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ABSTRACT 


The  U.S.  Army  Training  and  Doctrine  Command  (TRADOC)  Analysis  Center’s 
Joint  Dynamic  Allocation  of  Fires  and  Sensors  (JDAFS)  model,  a  low-resolution, 
Discrete  Event  Simulation  Model  with  embedded  optimization  enables  the  analysis  of 
many  scenarios  and  factors  to  explore  Joint  Intelligence,  Surveillance,  and 
Reconnaissance  (ISR)  missions.  JDAFS  is  a  powerful  model  that  combines  both  discrete 
event  simulation  and  the  optimization  of  a  linear  objective  function  to  generate  realistic, 
reasonable,  and  consistent  solutions  to  difficult  ISR  scheduling  problems.  Given  a 
scenario  and  a  mix  of  ISR  platforms,  JDAFS  optimizes  a  flight  schedule  and  executes  the 
missions.  This  research  develops  a  Joint  ISR  scenario,  explores  scenario  simulation 
results,  and  provides  a  proof-of-principle  analysis  that  aids  in  the  ISR  decision  making 
process. 

This  study  examines  274  design  points  in  each  of  two  scenarios,  a  non-penetrating 
scenario  that  allows  only  standoff  collection  and  a  penetrating  scenario  that  allows 
country  of  interest  overflight.  The  use  of  an  efficient  design  of  experiment  methodology 
enables  the  exploration  of  the  interior  and  exterior  of  the  response  surface  for  the  two 
experimental  scenarios.  Analysis  of  the  simulation  output  suggests  that  the  optimization 
interval  significantly  impacts  total  coverage.  In  the  non-penetrating  scenario,  shorter 
optimization  intervals  ensure  better  coverage;  however,  in  the  penetrating  scenario, 
longer  optimization  intervals  provide  for  improved  coverage.  The  disparity  is  explained 
by  reduced  likelihood  of  assignment  saturation  in  the  penetrating  scenario  due  to  the 
increased  number  of  mission  areas.  Sensor  range,  sensor  package  configuration,  and 
platform  dwell  time  also  affect  the  level  of  coverage.  This  is  clearly  demonstrated  by  the 
superior  coverage  provided  by  the  most  capable  ISR  platforms. 
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The  reader  is  cautioned  that  the  computer  programs  presented  in  this  research  may 
not  have  been  exercised  for  all  cases  of  interest.  While  every  effort  has  been  made, 
within  the  time  available,  to  ensure  that  the  programs  are  free  of  computational  and 
logical  errors,  they  cannot  be  considered  validated.  Any  application  of  these  programs 
without  additional  verification  is  at  the  risk  of  the  user. 
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EXECUTIVE  SUMMARY 


A  well  coordinated  and  synchronized  Intelligence,  Surveillance,  and 
Reconnaissance  (ISR)  program  is  essential  to  maintaining  global  situational  awareness 
and  when  necessary  enabling  a  response  with  decisive  force.  The  platforms  that  conduct 
ISR  missions  are  a  diverse  collection  of  low  density,  high  demand  assets  with  complex 
operating  requirements.  These  aircraft  must  be  operated  in  a  variety  of  environments  and 
are  subject  to  limitations  in  effective  range  and  deployment  locations.  To  further 
complicate  matters,  the  ability  of  sensors  to  collect  on  intelligence  targets  of  interest  is 
not  unlimited.  When  the  targets  are  beyond  the  range  of  the  collection  platforms  and 
sensors,  it  is  necessary  to  penetrate  the  airspace  of  sovereign  nations  to  gather  the 
required  intelligence.  This  is  typically  only  done  during  a  state  of  conflict  and  inherently 
brings  considerable  risks.  This  research  develops  a  Joint  ISR  scenario,  explores  scenario 
simulation  results,  and  provides  a  proof-of-principle  analysis  that  aids  in  the  ISR  decision 
making  process  to  give  the  responsive  edge  to  Combatant  Commanders. 

The  U.S.  Army  Training  and  Doctrine  Command  (TRADOC)  Analysis  Center’s 
Joint  Dynamic  Allocation  of  Fires  and  Sensors  (JDAFS)  model,  a  low-resolution, 
Discrete  Event  Simulation  Model  with  embedded  optimization  enables  the  analysis  of 
many  scenarios  and  factors.  This  thesis  identifies  critical  JDAFS  improvements  for  the 
representation  of  these  Joint  missions  and  explores  the  effects  of  21  factors  on  the 
Measures  of  Effectiveness  provide  by  the  model.  This  research  examines: 

1 .  What  are  the  trade-offs  associated  with  penetrating  versus  non-penetrating 
ISR  missions? 

2.  What  are  the  marginal  effects  of  increasing  or  decreasing  airborne  assets 
within  a  scenario? 

3.  What  are  the  appropriate  design  of  experiment  (DOE)  methodologies  and 
tools  for  penetrating  versus  non-penetrating  ISR  mission  analysis? 

4.  What  are  the  improvements  to  the  JDAFS  simulation  needed  to  credibly 
represent  the  Joint  ISR  mission? 

JDAFS  is  a  powerful  model  that  combines  both  discrete  event  simulation  and  the 
optimization  of  a  linear  objective  function  to  generate  realistic,  reasonable,  and  consistent 
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solutions  to  difficult  ISR  scheduling  problems.  Given  a  scenario  and  a  mix  of  ISR 
platforms,  JDAFS  optimizes  a  flight  schedule  and  executes  the  missions.  JDAFS 
considers  the  airframe  operational  parameters,  sensor  payload  capabilities,  base  and 
mission  area  locations,  and  line-of-sight  inputs  in  the  solution  to  the  ISR  scheduling 
problem. 

The  scenarios  built  for  this  study  are  designed  to  conduct  a  trade-off  analysis  for 
Joint  ISR  operations.  Two  types  of  scenarios  were  generated  for  examination  and 
comparison  within  JDAFS.  The  first  scenario  is  a  non-penetrating  scenario  where  the 
ISR  platforms  do  not  penetrate  the  Country  of  Interest’s  (COI)  national  airspace.  The 
internationally  accepted  buffer  of  22  kilometers  is  respected  on  all  flights  and  waypoints 
have  been  implemented  to  prevent  ingressing  and  egressing  aircraft  from  violating  the 
COI’s  sovereign  airspace.  The  second  scenario  assumes  that  conditions  have  changed  to 
allow  the  violation  of  the  COI’s  airspace.  With  the  incursions  into  the  COI’s  territory 
comes  the  risk  of  engagement  by  air  defense  assets,  in  this  case  surface-to-air  missiles 
(SAMs).  Figure  1  illustrates  the  penetrating  scenario  designed  for  this  experiment.  The 
only  difference  between  the  penetrating  scenario  and  the  non-penetrating  scenario  is  the 
inclusion  of  mission  areas  within  the  border  of  the  COI  for  the  penetrating  scenario. 

The  country  of  interest  for  the  scenario  is  depicted  as  a  hexagon  measuring 
1000km  between  any  of  its  widest  points.  The  size  of  the  country  allows  the  missions 
(ISR  targets)  to  be  widely  dispersed  and  ensures  that  even  the  most  capable  sensors 
included  in  the  study  do  not  cover  excessive  portions  of  the  country  from  a  single  mission 
area.  The  missions  are  dispersed  non-unifonnly  throughout  the  country  to  achieve  a 
sense  of  realism.  While  the  mission  dispersal  is  not  unifonn,  it  is  also  not  random.  Care 
is  taken  to  place  the  missions  such  that  the  various  mission  sensor  requirements  are 
widely  distributed  to  limit  disproportionate  coverage  by  a  single  platfonn.  Mission  areas 
were  initially  designed  to  ring  the  entire  COI;  however,  allowing  complete  access  to  all 
countries  surrounding  a  COI  may  be  unrealistic.  Overflight  of  mission  areas  along  the 
western  border  and  a  portion  of  the  southern  border  were  deemed  to  be  denied.  Four 
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operating  bases  for  the  ISR  platforms  were  placed  at  varying  distances  in  the  non-denied 
areas  surrounding  the  COI  to  allow  for  different  transit  times  from  base  to  mission  area. 
The  air  defense  sites  were  distributed  throughout  the  COI. 


Figure  1.  JDAFS  Scenario 

This  study  implements  an  experimental  design  technique  known  as  the  Nearly 
Orthogonal  Latin  Hypercube  (NOLH)  to  explore  how  changes  in  the  input  parameters 
(factors)  affect  the  model  output.  The  21  factors  to  be  varied  in  this  experiment  consist 
of  20  numbers  of  aircraft  at  a  base,  and  1  optimization  interval  factor.  The  20  aircraft 
factors  take  on  integer  values  between  0  and  3  or  0  and  6,  depending  on  the  type  of 
airframe.  The  optimization  interval  factor  is  a  continuous  variable  with  values  between 
0.5  and  6  hours. 

The  NOLH  design  provides  a  number  of  highly  desirable  qualities,  including 
space  filling,  orthogonality,  and  flexibility.  Space  filling  designs  provide  a  means  to 
explore  the  interior  and  exterior  regions  of  the  response  surface,  allowing  a  thorough 
examination  of  the  output  data  without  the  need  to  run  every  conceivable  design  point. 
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Orthogonality  ensures  independence  of  the  regressors,  or  coefficient  estimates  in 
resultant  regression  models.  The  NOLH  design  is  also  flexible,  allowing  rapid  changes 
to  input  factors,  factor  levels,  and  the  size  of  the  overall  design  space.  A  sophisticated 
NOLH  design  template  was  used  in  the  creation  of  the  experimental  design  for  this 
research. 

The  final  design  for  this  experiment  includes  274  design  points  to  explore  two 
scenarios;  a  non-penetrating  scenario  that  allows  only  standoff  collection  and  a 
penetrating  scenario  that  allows  country  of  interest  overflight.  The  ISR  platforms  are 
subject  to  attrition  by  air  defense  assets  in  the  penetrating  scenario  but  not  in  the  non¬ 
penetrating  scenario.  The  scenarios  each  include  25  collection  targets  broken  down  into 
55  individual  sensor  requirements.  An  analysis  of  the  output  data  from  the  two  scenarios 
executed  by  the  JDAFS  model  suggest: 

•  Optimization  is  the  most  significant  factor  in  both  the  scenarios,  but  has 
the  opposite  effect.  In  the  non-penetrating  scenario,  shorter  optimization 
intervals  are  better,  but  in  the  penetrating  scenario  longer  optimization 
intervals  yield  improved  coverage.  Increased  assignment  options  in  the 
penetrating  scenario  causes  this  disparate  behavior  by  the  optimization 
interval  factor. 

•  Coverage  is  not  necessarily  improved  by  adding  more  ISR  assets  in  the 
non-penetrating  scenario. 

•  Coverage  is  improved  by  the  addition  of  assets  in  the  penetrating  scenario. 
Coverage  would  likely  eventually  plateau  with  diminishing  returns  from 
the  addition  of  more  aircraft  in  this  scenario.  The  maximum  number  of 
airframes  in  the  scenario  was  not  sufficient  to  explore  this  behavior. 

•  Penetrating  ISR  platforms  average  20%  more  mean  coverage  than  non¬ 
penetrating  platforms. 

•  The  U-2  is  the  most  important  platfonn  factor  in  explaining  the  variability 
in  both  scenarios. 

•  The  RQ-4  results  in  the  majority  of  mission  coverage  for  both  scenarios 
due  to  its  ability  to  remain  on  station. 

NOTE:  The  ISR  platforms  and  sensors  used  in  this  study  and  their 
operational  parameters  and  capabilities  DO  NOT  represent  actual  classified 
real  world  performance  characteristics. 
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In  addition  to  exploring  the  output  from  the  simulation,  JDAFS  was  exercised  to 
evaluate  additional  changes  that  would  be  required  to  better  model  Joint  ISR  missions. 
Several  recommended  additions  or  improvements  were  identified: 

•  Allow  ISR  platforms  to  receive  credit  for  collection  on  targets  within 
sensor  range  during  ingress  and  egress  to  mission  areas. 

•  Create  probability  distributions  to  represent  current  and  future  force 
sensors  from  certified  data  to  accurately  represent  the  acquisition  of 
targets  by  sensors  in  the  model. 

•  Implement  revisit  valuation  schemes  for  missions  to  more  credibly 
represent  ISR  mission  characteristics. 

•  Addition  of  joint  assets  (platforms,  sensors,  munitions,  etc.)  in  the  JDAFS 
database  for  reuse  in  joint  studies. 

•  Further  develop  the  functionality  of  JDAFS  to  more  credibly  represent 
Joint  Sensors  to  include  satellite  assets. 

•  Provide  a  more  robust  representation  of  unmanned  aerial  vehicles  with 
weapons  and  sensors  that  allow  force-on-force  analysis  of  differing 
platform  mixes. 

•  Develop  an  integrated  design  of  experiments  interface  that  enables  a  quick 
determination  of  alternatives. 

•  Incorporate  the  InputGenerator  and  OutputGenerator  scripts  into  the 
JDAFS  program.  The  integration  of  these  tools  will  eliminate  the  need  for 
the  pre  and  post-processing  of  data  and  eliminate  the  need  to  transfer 
excessively  large  files. 

•  Develop  a  simple,  well  documented  JDAFS  interface  to  allow  an  analyst 
to  easily  enter  the  data  for  the  various  input  factors. 

•  Develop  a  comprehensive  JDAFS  Users/ Analyst  Manual. 
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I.  INTRODUCTION 


A.  BACKGROUND 

The  national  security  of  the  United  States  faces  dynamic  challenges  across  the 
globe.  A  well  coordinated  and  synchronized  Intelligence,  Surveillance,  and 
Reconnaissance  (ISR)  program  is  essential  to  maintaining  global  situational  awareness 
and  when  necessary  enabling  a  response  with  decisive  force.  “We  know  that  if  the 
intelligence  cycle  lags  behind  operational  and  command  decision-making  windows  it  is 
not  actionable.  Just  as  a  maneuver  fonnation  on  the  ground,  at  sea  or  in  the  air  is  a 
warfighting  system,  so  is  intelligence.  By  making  ISR  the  responsibility  of  a  functional 
Combatant  Commander  we  gain  synergy  and  perspective  among  warfighting  systems” 
(U.S.  Congress  2007,  7).  The  outcome  of  this  research  provides  tools  that  aid  in  the  ISR 
decision  making  process  for  the  USSTRATCOM  Global  Defense  Posture  and  give  the 
responsive  edge  to  Combatant  Commanders. 

The  need  to  plan  and  act  globally  is  more  important  than  ever  before.  Writing  on 
the  shift  from  a  regional  to  a  global  perspective,  General  Myers  (2003),  as  Chairman  of 
the  Joint  Chiefs  of  Staff,  noted  that, 

We  will  need  global  ISR  activities  for  gathering  indications  and  warning 
data  and  for  otherwise  enabling  global  strike,  space  operations,  certain 
elements  of  10,  and  integrated  missile  defense.  Moreover,  we  need  global 
C2  capabilities  to  enable  integrated  global  missile  defense,  facilitate  global 
strike,  integrate  regional  operations  with  global  operations,  and  integrate 
regional  operations  in  one  AOR  with  those  of  another.  Knitting  together 
various  regionally  focused  ISR  activities  is  unlikely  to  yield  a  coherent 
global  perspective.  Simply  put,  we  cannot  obtain  a  relevant  global 
perspective  without  ISR  activities  that,  to  some  degree,  are  globally 
coordinated  and  directed — a  function  performed  by  the  Defense 
Intelligence  Agency.  The  new  factor  is  that,  given  the  low-density/high- 
demand  nature  of  many  of  our  ISR  resources,  regional  combatant 
commands  are  more  likely  than  before  to  be  required  to  conduct  ISR 
activities  in  support  of  global  operations. 
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Reflecting  the  shift  in  perspective,  Change  Two  to  the  Unified  Command  Plan 
designated  US  Strategic  Command  (USSTRATCOM)  as  the  executive  agent  for  the 
Global  ISR  mission.1  The  assignment  of  the  Global  ISR  mission  to  USSTRATCOM 
ensures  that  a  comprehensive,  global  approach  is  applied  to  provide  timely,  survivable, 
persistent  coverage  of  key  targets  or  areas  of  interest. 

The  challenge  of  meeting  the  nation’s  ISR  demands  is  enormous.  The  platforms 
that  conduct  ISR  missions  are  a  diverse  collection  of  low  density,  high  demand  assets 
with  complex  operating  requirements.  These  aircraft  must  be  operated  in  a  variety  of 
environments  and  are  subject  to  limitations  in  effective  range  and  deployment  locations. 
To  further  complicate  matters,  the  ability  of  sensors  to  collect  on  intelligence  targets  of 
interest  is  not  unlimited.  When  the  targets  are  beyond  the  range  of  the  collection 
platforms  and  sensors,  it  is  necessary  to  penetrate  the  airspace  of  sovereign  nations  to 
gather  the  required  intelligence.  This  is  typically  only  done  during  a  state  of  conflict  and 
inherently  brings  considerable  risks. 

A  need  exists  to  evaluate  the  trade-offs  based  on  various  ISR  platfonn  capabilities 
and  basing  configurations.  The  Joint  Dynamic  Allocation  of  Fires  and  Sensors  (JDAFS) 
model,  a  low-resolution,  Discrete  Event  Simulation  Model  with  embedded  optimization 
enables  the  analysis  of  many  scenarios  and  factors  in  a  relatively  short  time  period  when 
compared  to  other  simulations  (U.S.  Army  Training  and  Doctrine  Command  2007a). 

This  research  is  sponsored  by  the  U.S.  Army  Training  and  Doctrine  Command 
(TRADOC)  Analysis  Center  (TRAC),  Monterey.  Collocated  with  the  Naval 
Postgraduate  School,  TRAC-Monterey  is  focused  on  research  in  computer  simulation 
technology  and  military  operations  research.  “TRAC  Monterey  is  recognized  as  a 
premier  applied  research  organization  for  military  modeling,  simulation,  methodologies, 
and  analysis”  (U.S.  Army  Training  and  Doctrine  Command  2007b). 


1  President  Bush  signed  Change  Two  to  the  Unified  Command  Plan  on  Jan.  10,  2003,  and  tasked 
USSTRATCOM  with  four  previously  unassigned  responsibilities:  global  strike,  missile  defense  integration, 
Department  of  Defense  Information  Operations,  and  C4ISR  (command  and  control,  communications, 
computers,  intelligence,  surveillance,  and  reconnaissance)  (U.S.  Strategic  Command,  2007). 
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B.  PURPOSE  OF  RESEARCH 

The  purpose  of  this  research  is  to  simulate,  by  use  of  the  JDAFS  model,  the 
effects  of  varying  key  factors  associated  with  the  operation  and  employment  of  manned 
and  unmanned  ISR  platforms.  The  main  focus  of  this  research  is  an  analysis  of  the  trade¬ 
offs  associated  with  specific  platforms  and  basing  decisions  on  the  execution  of 
penetrating  vs.  non-penetrating  missions.  The  analytical  findings  provide  a  foundation 
for  future  ISR  scheduling  and  requirements  decisions.  Additionally,  prior  to 
commencing  production  runs,  the  JDAFS  model  will  be  exercised  to  identify  any 
problems  or  deficiencies  that  would  prevent  the  use  of  this  model  for  the  planned 
research. 

C.  RESEARCH  QUESTIONS 

The  flexibility  and  power  of  the  JDAFS  model  to  simulate  dynamic  ISR  missions 
offers  a  number  of  areas  for  potential  research.  This  research  will  explore: 

1 .  What  are  the  trade-offs  associated  with  penetrating  versus  non-penetrating 
ISR  missions? 

2.  What  are  the  marginal  effects  of  increasing  or  decreasing  airborne  assets 
with  in  a  scenario? 

3.  What  are  the  appropriate  design  of  experiment  (DOE)  methodologies  and 
tools  for  penetrating  versus  non-penetrating  ISR  mission  analysis? 

4.  What  are  the  improvements  to  the  JDAFS  simulation  needed  to  credibly 
represent  the  Joint  ISR  mission? 

D.  SCOPE  OF  RESEARCH 

The  Design-of-Experiments  (DOE)  and  JDAFS  simulation  are  developed  based 
on  two  fictional,  yet  realistic  scenarios  using  data  acquired  from  unclassified  sources.  In 
establishing  the  DOE,  specific  factors  that  contribute  to  the  ISR  platfonn  attributes  of 
timely,  survivable,  and  persistent  are  identified.  The  identified  factors  are  examined  at 
various  ranges  within  the  DOE  framework.  However,  the  number  of  factors  involved 
within  a  full-factorial  design  makes  examining  every  combination  of  inputs  impractical. 


3 


Therefore,  an  efficient,  robust  DOE  is  employed  to  examine  the  response  surface.  Output 
from  the  simulation  runs  is  analyzed  to  evaluate  how  the  various  input  factors  affect  the 
predetennined  Measures-of-Effectiveness  (MOE)  across  the  range  of  scenarios. 


E.  THESIS  ORGANIZATION 

Chapter  II  provides  a  history  and  description  of  the  Joint  Dynamic  Allocation  of 
Fires  and  Sensors  (JDAFS)  model.  Chapter  III  is  an  overview  of  the  scenarios  to  be 
executed  along  with  a  listing  of  the  various  manned  and  unmanned  aircraft  and  sensors 
modeled  in  the  simulation.  Chapter  IV  covers  the  design  of  experiments  to  facilitate  the 
running  of  the  JDAFS  model.  Chapter  V  details  the  analysis  completed  on  the  model 
output  data.  Chapter  VI  is  a  summary  of  findings  based  on  the  research  conducted  and 
includes  recommendations  for  follow-on  study. 
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II.  JDAFS  MODEL,  INPUTS  AND  METHODOLOGY 


A.  HISTORY  OF  THE  JDAFS  MODEL 

The  Dynamic  Allocation  of  Fires  and  Sensors  (DAFS)  model  was  created  for 
TRAC-Monterey  in  2002  (Havens  2002).  DAFS  is  a  low-resolution,  constructive  entity- 
level  simulation  framework  programmed  in  JAVA  and  incorporates  the  functionality  of 
the  Simkit  simulation  toolkit  created  by  Dr.  Arnold  Buss  (2006).  The  original  version  of 
the  model  provided  a  low-resolution  simulation  tool  for  TRAC-Monterey  to  explore 
scenarios  for  the  Army’s  Future  Combat  System.  JDAFS  has  undergone  substantial 
revision  and  expansion  by  TRAC-Monterey  to  improve  the  functionality  and  usability  of 
the  model  as  an  analysis  tool.  In  early  2007,  the  model  name  was  changed  to  Joint 
Dynamic  Allocation  of  Fires  and  Sensors  (JDAFS)  in  recognition  that  the  model’s 
functionality  had  expanded  beyond  use  for  purely  Army  studies. 

Between  its  original  creation  in  2002  and  early  2007,  DAFS  has  evolved  into  two 
separate  models: 

•  Joint  Dynamic  Allocation  of  Fires  and  Sensors  (JDAFS),  which  consisted 
primarily  of  the  simulation  and  visualization  portion  of  the  program 

•  Assignment  Scheduling  Capability  for  Unmanned  Aerial  Vehicles  (ASC- 
U)  tool,  which  retained  the  scheduling  and  optimization  functionality. 

As  part  of  this  effort,  the  two  programs  have  subsequently  been  recombined  in  the 
current  version  of  JDAFS. 

B.  JDAFS  ORIGINAL  CONFIGURATION 

ASC-U  and  JDAFS  were  developed  from  DAFS  but  were  two  separate  software 
development  branches.  ASC-U  provided  a  robust  scheduling  capability  for  manned  and 
unmanned  aerial  sensor  platforms  using  optimization  to  make  assignments  using  an 
approximate  dynamic  programming  approach.  ASC-U  was  detenninistic  since  no 
stochastic  effects  were  considered  by  the  methodology.  DAFS  provided  a  low-resolution 
approach  to  force-on-force  combat  and  relied  on  its  optimization  capability  for  fires 
assignment.  DAFS  represented  the  stochastic  effects  of  the  fires  actions. 
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1.  JDAFS  Functionality 

JDAFS,  in  its  original  configuration,  was  an  entity  level  simulation  that  relied  on 
scripted  scenarios  to  execute  a  simulation.  Fires  allocation  was  conducted  throughout  the 
scenario  via  an  optimization  algorithm  based  on  matching  available  weapons  and  firing 
platforms  to  targets,  dependent  on  acquisition  sensor  input.  The  adjudication  of  fires  was 
a  stochastic  process  driven  by  a  pre-determined  distribution.  Once  a  simulation 
commenced,  the  scenario  would  run  to  completion  as  scripted  with  no  ability  for  dynamic 
re-tasking. 

2.  Event  Graph  Methodology 

The  DAFS  discrete  event  simulation  model  is  built  on  the  concept  of  Schruben’s 
(1983)  event  graph  methodology.  The  event  graph  approach  is  uniquely  suited  to 
describe  and  implement  the  types  of  movement,  sensing,  and  weapons  effects  interactions 
required  in  a  low-resolution  simulation.  The  use  of  event  graph  methodology  can  provide 
significant  advantages  over  a  time  stepped  approach  with  respect  to  model  run  time. 

The  foundation  of  event  graph  methodology  is  the  event  list.  Discrete  event 
simulations  are  run  by  sequentially  executing  events  on  the  event  list.  Buss  and  Ahner 
(2006b,  1353)  provide  the  following  explanation  of  the  event  graph: 

An  Event  Graph  consists  of  nodes  and  directed  arcs.  Each  node 
corresponds  to  an  event,  or  state  transition,  and  each  arc  corresponds  to  the 
scheduling  of  other  events.  Each  arc  can  optionally  have  an  associated 
boolean  condition  and/or  a  time  delay.  Figure  2  shows  the  fundamental 
construct  for  Event  Graphs  and  is  interpreted  as  follows:  the  occurrence  of 
Event  A  causes  Event  B  to  be  scheduled  after  a  time  delay  of  t,  providing 
condition  (/)  is  true  (after  the  state  transitions  for  Event  A  have  been 
performed).  By  convention,  the  time  delay  t  is  indicated  toward  the  tail  of 
the  scheduling  edge  and  the  edge  condition  is  shown  just  above  the  wavy 
line  through  the  middle  of  the  edge.  If  there  is  no  time  delay,  then  t  is 
omitted.  Similarly,  if  Event  B  is  always  scheduled  following  the 
occurrence  of  Event  B,  then  the  edge  condition  is  omitted,  and  the  edge  is 
called  an  unconditional  edge.  Thus,  the  basic  Event  Graph  paradigm 
contains  only  two  elements:  the  event  node  and  the  scheduling  edge  with 
two  options  on  the  edges  (time  delay  and  edge  condition). 
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Figure  2.  Event  Graph  Example  (From  Buss  and  Ahner  2006b) 

For  additional  infonnation  on  event  graph  mythology  as  it  applies  to  modeling  movement 
and  detection,  see  Simple  Movement  and  Detection  in  Discrete  Event  Simulation  by  Buss 
and  Sanchez  (2005). 

C.  ASC-U  ORIGINAL  CONFIGURATION 

The  scheduling  of  ISR  platfonns  is  an  inherently  dynamic  and  complex  process. 
The  standalone  ASC-U  program  is  a  powerful  scheduling  tool  that  combines  both 
simulation  and  optimization  to  help  overcome  these  complexities.  Provided  with  high 
quality  input  data,  ASC-U  is  capable  of  generating  realistic,  reasonable,  and  consistent 
solutions  to  difficult  ISR  scheduling  scenarios.  (Ahner  et  al.  2006b,  Nannini  2006) 

Despite  using  simulation  to  determine  its  solutions,  ASC-U  is  not  a  stochastic 
model.  It  produces  identical  flight  schedules  for  a  given  set  of  input  parameters.  The 
simulation  portion  of  the  model  overcomes  the  discrete  time  step  artificiality  that  can  be 
introduced  by  multi-period  optimizations  (Ahner  et  al.  2006a). 

While  ASC-U  is  primarily  designed  for  the  scheduling  of  UAV  missions,  it  easily 
incorporates  mixes  of  manned  and  unmanned  aircraft  to  solve  the  following  challenge 
(Ahner  et  al.  2006a,  2): 

Given  a  scenario  that  specifies  the  number  of  each  type  of  UAV,  initial 
UAV  locations,  and  UAV  perfonnance  characteristics,  detennine  the 
number  of  missions  that  can  successfully  be  completed  and  the  schedule 
for  each  UAV.  The  solution  must  consider  ground  control  station  (GCS) 
locations  and  capacities,  launch  and  recovery  site  (LRS)  locations  and 
capacities... and  communication  platform  footprint  and  capacities. 

The  ultimate  schedule  derived  by  ASC-U  is  driven  by  calculating  the  maximum  total 
mission  value  based  on  the  performance  parameters  of  the  available  ISR  platforms. 
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1.  Primary  ASC-U  Components 

ASC-U’s  primary  components  consist  of  missions,  ISR  platforms,  sensor 
packages,  LRSs,  GCSs.  Each  component  is  populated  in  a  Microsoft  Access  database 
using  one  or  more  tables  for  the  various  elements. 

a.  Missions 

Missions,  or  ISR  targets,  consist  of  a  set  of  coordinates  combined  with  a 
set  of  sensor  requirements.  For  example,  a  surface-to-air  missile  site  would  have  a  set  of 
locating  coordinates  and  require  coverage  by  electro-optical/infrared  (EO/IR)  sensors, 
synthetic  aperture  radar  (SAR),  and  signals  intelligence  (SIGINT)  sensors.  In  addition, 
each  sensor  requirement  is  assigned  a  value  rate.  ISR  platforms  receive  credit  or  value  by 
being  on-station  within  sensor  range  of  a  sensor  requirement  provided  that  the  mission,  or 
sensor  requirement,  is  not  already  being  covered  by  another  platfonn.  The  total  value 
derived  is  found  by  multiplying  the  value  rate  of  the  sensor  requirement  by  the  ISR 
platform’s  time  on-station.  ISR  platfonns  that  carry  multiple  sensor  packages  can  fulfill 
multiple  mission  requirements.  The  value  achieved  by  satisfying  multiple  mission 
requirements  is  additive  (Ahner  et  al.  2006b). 

b.  ISR  Platforms 

Individual  ISR  platforms  in  ASC-U  consist  of  a  unique  name,  an 
associated  LRS,  a  transition  time,  an  airspeed,  an  operational  endurance,  a  GCS  operating 
radius,  and  one  or  more  sensor  types.  Unique  names  automatically  generate  for  the 
individual  ISR  platforms  to  allow  mission  assignments,  the  tracking  or  sensor 
requirements  accomplished,  and  the  state  of  the  platform  within  the  simulation.  An  ISR 
platform’s  airspeed  and  operational  endurance  determine  feasibility  of  assignment  to 
specific  missions  and  the  amount  of  time  the  aircraft  can  remain  on-station  prior  to 
returning  to  an  LRS.  A  platform’s  transition  time  accounts  for  the  minimum  time  an 
aircraft  must  spend  at  an  LRS  for  maintenance  and  refueling  prior  to  launching  for  a  new 
mission.  The  GCS  operating  radius2  dictates  the  maximum  distance  a  UAV  ISR  platform 

2  GCS  Control  Radius  was  assumed  to  infinite  in  this  study.  All  communications  with  UAVs  were 
deemed  to  occur  via  satellite  and  GCS  control  is  not  required  for  manned  platforms. 
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can  travel  from  a  GCS  without  losing  contact.  Sensor  types  assigned  to  individual 
platforms  detennine  the  types  of  missions  capable  of  being  covered  by  specific  aircraft 
(Ahner  et  al.  2006b,  1350). 

c.  Sensor  Packages 

Each  ISR  platform  has  an  associated  sensor  package.  These  packages 
consist  of  sensor,  weapon,  or  communications  capabilities.  These  capabilities  control  the 
types  of  missions  that  are  assignable  to  the  various  platfonns  (Ahner  et  al.  2006b,  1350). 

d.  LRS 

LRSs  provide  an  operating  base  from  which  the  ISR  platforms  operate. 
An  LRS  has  a  unique  name,  an  associated  ISR  platform  type,  a  set  of  locating 
coordinates,  and  a  capacity.  While  not  used  within  the  context  of  this  thesis,  LRSs  have 
the  ability  to  move  if  associated  with  UAVs  (Ahner  et  al.  2006b,  1350). 

e.  GCS 

The  GCS  provides  a  means  for  controlling  UAV  platforms  during  flight. 
Like  an  LRS,  a  GCS  is  composed  of  a  unique  name,  an  associated  ISR  platfonn  type,  a 
set  of  locating  coordinates,  and  a  capacity.  Also,  just  as  with  LRSs,  GCSs  have  the 
ability  to  move  during  a  scenario  (Ahner  et  al.  2006b,  1350). 

2.  Simulation  and  Optimization 

Using  the  input  data  from  the  components  described  above,  ASC-U  uses  the 
combination  of  optimization  and  simulation  to  deliver  quality  solutions  to  complex 
scheduling  and  assignment  problems  within  an  approximate  dynamic  programming 
framework.  ASC-U  is  built  upon  the  DAFS  discrete  event  simulation  model  that  contains 
an  embedded  constrained  value  optimizer  (CVO). 

The  CVO  algorithm  delivers  a  myopic  optimal  solution.  The  myopic,  or  locally 
optimal,  solution  is  necessary  to  maintain  problem  tractability.  The  calculation  of  a 
globally  optimal  solution  would  become  computationally  infeasible  for  all  but  the 
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smallest  of  problems.  The  CVO  derives  its  solution  by  comparing  the  ISR  resources 
available  to  the  existing  sensor  requirements  to  determine  the  optimal  scheduling  to 
maximize  total  value  (Ahner  et  al.  2006b,  1351). 

3.  ASC-U  Platform  Assignment  Process 

The  following  description  from  the  ASC-U  Users/ Analyst  Manual  illustrates  the 
ASC-U  platform  assignment  process  (Ahner  et  al.  2006a,  4-5): 

For  example,  consider  2  UAVs  assigned  to  4  missions  as  depicted  in 
Figure  3.  ASC-U  performs  optimal  assignments  at  predetermined  time 
intervals  (1).  At  time  to,  assume  two  UAVs  are  available,  UAV1  and  UAV 
2.  The  tool  considers  all  available  UAVs  with  available  GCS  control  and 
all  missions  with  value  in  the  fixed  time  horizon  time  window  (missions  1 , 

2,  3  but  not  4  for  time  to)  (2).  A  UAV  is  assigned  to  a  mission  and  can 
service  any  sensor  requirement  associated  with  that  mission  if  it  has  the 
correct  sensor.  Assume  the  optimal  assignment  is  UAV  1  assigned  to 
Mission  1  and  UAV  2  assigned  to  Mission  2.  UAV  1  is  launched 
immediately  to  arrive  at  Mission  1  as  soon  as  possible  (3).  UAV  2  does 
not  launch  but  is  scheduled  for  launch  to  arrive  just-in-time  for  the 
beginning  of  Mission  2.  At  time  fi,  UAVs  that  are  scheduled  to  launch  but 
have  not  yet  launched  are  “unassigned”  and  considered  for  assignment 
(UAV  2  is  unassigned)  (4).  UAV  2  is  again  assigned  to  Mission  2  and 
scheduled  for  launch  to  arrive  just-in-time.  UAV  assignments  do  not 
change  once  they  have  been  launched. 
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The  process  ASC-U  goes  through  in  the  construction  of  flight  schedules  is 
illustrated  in  Figure  4.  Beginning  in  the  Dynamic  Simulation  State  Transitions  box  and 
proceeding  clockwise,  the  system  is  prompted  to  execute  an  optimization.  The  Value  of 
Potential  Assignment  Generator  detennines  the  potential  value  that  could  be  accumulated 
within  the  allotted  time  horizon  for  all  possible  ISR  platform  assignments.  The  value 
derived  at  this  stage  accounts  for  all  of  the  parameters  affecting  the  aircraft,  including 
travel  time,  time-on-station,  and  GCS  capacity.  The  solution  from  the  Value  of  Potential 
Assignment  Generator  is  passed  to  the  optimization  routine.  The  optimization  stage 
returns  the  locally  maximal  value  of  the  maximization  problem.  This  solution  is  passed 
to  the  UAV  scheduler  for  platform  assignment.  The  aircraft  are  slated  to  launch  such  that 
they  arrive  promptly  at  the  mission  start  time,  taking  travel  time  into  account.  Following 
the  preset  re-optimization  time  interval  the  process  reenters  the  Dynamic  Simulation 
State  Transition  phase  and  the  process  continues  until  a  predetermined  termination  time  is 
met  (Ahner  et  al.  2006a). 


The  value  of  each  assignment  is  determined  BEFORE  the  optimization 
taking  into  account  travel  time,  time-on-station,  LRS  location  over  time, 
and  GCS  location  and  capacity  over  time. 


Values  of  assignments  are  passed  to  Optimization. 

Value  of  Potential 
Assignment  Generator 


^  Optimization 


Periodic  or  event  triggered 
reoptimization 


V 


Optimization  provides  “optimal” 
assignment  as  cues  for  where 
airframes  should  go. 


Dynamic  Simulation 
State  Transitions 


UAV  Scheduler 


The  UAV  Scheduler  determines  when  airframes  should 
be  launched  to  locations  given  to  it  by  the  optimization. 


Figure  4.  ASC-U  Dynamic  Cueing  and  Transitions  (From:  Ahner  et  al.  2006a) 
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For  additional  details  on  installing  and  using  the  ASU-C  tool,  see  the  ASC-U 
Users/ Analysts  Manual. 

At  every  optimization  event  the  CVO  for  the  three-dimensional  (3D)  integer 
program  (IP)  must  be  solved.  LP  Solve  5.5,  programmed  in  C,  is  implemented  in 
JDAFS  and  accessed  using  the  Java  Native  Interface  (JNI)  (Ahner  et  al.  2006b,  1351). 
This  solver  algorithm  is  an  implementation  of  the  simplex  method  using  the  branch  and 
bound  technique  to  solve  linear  program  and  integer  program  formulations.  The  IP 
formulation  in  ASC-U  is  as  follows: 


Indices: 

i  e  MP 

MissionPackage:  A  package  with  capabilities  that  is 
installed  on  a  MissionPlatform  when  the  MissionPlatform 
is  launched. 

jeMA 

Mission  Area:  A  location  where  a  MissionPlatfonn  with  a 
MissionPackage  patrols. 

k  e  Msn 

Mission:  A  location  that  requires  coverage  by  a 
MissionPackage. 

leLRS 

LRS:  A  location  that  holds  MissionPackages  and 
MissionPlatfonns  that  are  not  currently  in  use. 

g  e  GCS 

GCS:  A  ground  control  station. 

Sets: 

AvailPackage, 

The  set  of  available  MissionPackages  at  LRS  1 

Areas 

The  set  of  MissionAreas  assigned  to  GCSg 

( Assignment  made  by  a  heuristic  prior  to 
optimization ) 

Data: 

ready  _  platform , 

The  number  of  MissionPlatfonns  ready  for  launch 
at  LRSi. 

capacity 

The  remaining  GCS  capacity  at  GCSg. 

costToLaunch 

The  cost  of  assigning  a  MissionPackage  to  a  MissionArea. 

Cm 

The  value  gained  by  assigning  MissionPackage  i  to 
MissionArea  j  to  cover  Mission  k. 
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Variables: 

Xjjk  1  if  MissionPackage  i  is  assigned  to  MissionArea  j  to  cover 

Mission  k,  0  otherwise. 

Y..  1  if  MissionPackage  i  is  assigned  to  MissionArea  j,  0 

otherwise. 

Formulation: 

1.  Max^Cijk*  Xjk  - costToLaunch^Yj 

ijk  i] 

Subject  to: 

2-  Y„  -  EX,  v,y 

k 

3-  X  ■  Y,  lijk 

4.  Vi 

j 

5-  ZV,»  5  1  Vi- 

ij 

6.  ^  Yy  <  ready  _  platform ,  V/ 

i  eAvailPackagej  j 

2-  IE  Yy  <  capacity  g  Vg 

i  jeAreag 

8.  Xljk  e  (0,1) 

9.  lis(0,l) 

Explanation: 

1 .  Maximize  the  total  values  of  the  assignments 

2.  Force  Kto  be  0  if  no  Mission  is  covered  by  assigning  MissionPackage  i  to 
MissionArea  j. 

3.  Force  F.  to  be  1  if  any  MissionPackage  i  is  assigned  MissionArea  j. 

4.  Each  MissionPackage  is  assigned  to  at  most  one  MissionArea. 

5.  Each  Mission  is  covered  by  at  most  one  MissionPackage. 

6.  The  number  of  MissionPackages  from  an  LRS  assigned  does  not  exceed 
the  number  of  MissionPlatforms  ready  for  launch  at  the  LRS. 

7.  The  number  of  MissionPackages  assigned  to  MissionAreas  covered  by 
each  GCS  does  not  exceed  the  remaining  GCS  capacity. 
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8.  Ensure  that  a  MissionPlatfonn  is  assigned  to  only  one  MissionArea  at  a 
time. 

9.  Ensure  that  a  MissionPackage  is  assigned  to  only  one  MissionArea  at  a 
time. 

Cijk  calculations: 

Cyk  will  be  zero  if  MissionPackage  i  does  not  have  the  needed  capability  to  cover 
Mission  k  or  Mission  k  is  not  within  range  or  line-of-sight  of  MissionPackage  i 
when  at  MissionArea  j.  Otherwise,  Qa  is  the  total  value  remaining  for  Mission  k 
during  the  period  that  MissionPackage  i  can  be  on-station  at  MissionArea  j,  taking 
into  account  the  endurance  of  the  MissionPlatfonn  carrying  the  MissionPackage 
and  the  ingress/egress  time  from  the  LRS  at  which  the  MissionPackage  is  located. 

Optimized  schedules  are  ‘flown’  by  the  simulation  portion  of  the  DAFS  model  in 
order  to  capture  the  dynamics  of  the  platfonn  transitions  and  movement.  The  simulation 
functionality  takes  on  increased  importance  in  scenarios  where  fires  and  complex 
relationships  between  moving  GCSs  and  LRSs  are  involved. 

D.  CHANGES  IMPLEMENTED  IN  JDAFS 

To  maximize  the  usefulness  of  JDAFS  as  analytical  tool,  several  changes  are 
necessary.  These  improvements  involve  restoring  some  of  the  capabilities  from  the 
original  model,  as  well  as,  adding  new  functionality.  The  following  modifications  are  the 
initial  changes  identified  to  make  the  simulation  viable  for  this  study;  further  operational 
testing  of  JDAFS  is  necessary  to  identify  any  remaining  bugs  and  deficiencies  prior  to 
conducting  production  runs. 

1.  Scheduling  Tool 

To  simulate  an  ISR  scenario  from  start  to  finish,  some  means  of  creating  a  flight 
schedule  or  Air  Tasking  Order  (ATO)  is  required.  While  several  options  were 
considered,  including  building  a  fixed  schedule  prior  to  execution  or  the  creation  of  a 
heuristic  scheduling  method,  the  best  alternative  was  to  integrate  ASC-U  back  into 
JDAFS.  This  created  not  only  the  ability  to  build  schedules  on  the  fly,  but  also  the  means 
to  optimize  the  flight  schedule  as  the  simulation  progressed.  In  addition,  ASC-U, 
coupled  with  pre-calculated  line-of  sight  detenninations  between  mission  and  mission 


areas  solves  a  two  stage  assignment  problem. 
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2. 


Mission  Areas 


The  creation  of  mission  areas  was  required  to  provide  a  method  to  simulate  an 
ISR  platform  on-station.  As  a  low-resolution  simulation,  JDAFS  does  not  model  flight 
characteristics  that  would  be  included  in  a  higher  resolution  model  such  as  turning, 
climbing,  descending,  speed  changes,  and  other  more  sophisticated  perfonnance 
parameters.  In  JDAFS,  platforms  actually  transit  from  a  base  to  a  mission  area  and 
remain  stationary  at  that  mission  area  for  the  duration  of  the  sortie.  This  artificiality  is  a 
trade-off  between  functionality  in  the  form  of  minimizing  solution  times  necessary  for  the 
execution  of  the  model  and  realism. 

3.  Bases 

The  integration  of  ASC-U  brings  with  it  the  ability  to  create  Launch  and 
Recovery  Sites  (LRS)  and  Ground  Control  Stations  (GCS).  Virtual  operating  bases  can 
be  established  by  setting  a  common  set  of  coordinates  for  these  entities  and  associating 
specific  ISR  platfonns.  ASC-U  has  the  ability  to  account  for  mobile  GCSs  and  LRSs, 
however,  for  this  study,  all  GCSs  and  LRSs  with  identical  coordinates  are  treated  as  fixed 
operating  bases. 

E.  IMPROVED  JDAFS  CONFIGURATION 

The  current  version  of  JDAFS  is  a  more  robust  version  of  the  original  program.  It 
incorporates  all  of  the  functionalities  of  the  ASC-U  program  described  above  along  with 
a  richer  simulation  capability.  The  remainder  of  this  section  addresses  specific  details 
related  to  the  description  and  execution  of  the  improved  JDAFS  configuration. 

1.  LOS  Calculation 

The  minimum  altitude  line-of-sight  (LOS)  between  all  missions  and  mission  areas 
is  pre-calculated  prior  to  the  execution  of  any  JDAFS  scenario.  The  predetermination  of 
the  minimum  altitude  LOS  is  what  enables  3D  optimization  in  the  execution  of  JDAFS 
scenarios.  For  the  purposes  of  this  study,  minimum  altitude  (LOS)  is  calculated  in  two 
steps.  First,  the  distance  between  the  mission  and  mission  area  is  calculated  as  follows: 
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Distance  =  J(x  -x 

V  V  mission  mission  area 


)  ^  (  y  mission  T mission  area  ) 


Second,  the  minimum  altitude  required  to  achieve  LOS  is  calculated3: 

(Distance)2 


Minimum  Altitude  =  ■ 


2x6378155 


The  results  of  the  LOS  calculations  are  then  used  to  populate  the  MinLOSAlt  field  in  the 
LOS  input  table. 


Figure  5.  Minimum  Altitude  Line-of-Sight  Determination 

The  scenarios  in  this  study  assume  flat  terrain  with  only  the  curvature  of  the  earth 
affecting  the  ability  to  achieve  line-of-sight.  The  inclusion  of  manmade  or  terrestrial 
terrain  features  obstructing  LOS  are  beyond  the  scope  of  this  study.  For  the  execution  of 
JDAFS  scenarios  based  on  real  world  geographic  locations,  a  LOS  calculation  tool  has 
been  created  by  TRAC-MTRY.  The  LOS  tool  uses  Digital  Terrain  Elevation  Data 
(DTED)  to  calculate  the  minimum  altitude  LOS  based  on  the  terrain  at  and  around  the 
actual  coordinates  of  the  missions  and  mission  areas. 

2.  Input  Requirements 

JDAFS  runs  on  an  XML  file  created  from  inputs  in  an  Access  database.  The 
Access  database  contains  the  various  tables  required  for  the  execution  of  a  scenario. 


3  The  minimum  altitude  calculation  is  derived  from  the  equation  for  detennining  the  distance  to  the 
visual  horizon:  Distance  to  the  visual  horizon  R  =  ( 2  •  Rearth  • h )  where  h  is  the  sensor  height,  in  meters, 
and  Rearth  is  the  radius  of  the  earth  at  the  equator,  6378155  meters  (Wittenberg  1997). 
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Only  the  data  tables  used  for  this  study  will  be  discussed  further.  For  information  on  the 
complete  list  of  tables  and  their  uses  see  the  ASC-U  User’s/ Analyst  Manual  (Ahner  et  al. 
2006b)  and  the  JDAFS  User’s  Guide  (Ahner  et  al.  2006c).4’ 5 

a.  DAFSScenario 

The  DAFS  Scenario  table  is  the  primary  driver  for  the  simulation.  Key 
elements  determined  in  this  table  include  the  optimization  interval,  the  number  of 
replications  to  be  executed,  and  the  length  of  the  scenario  to  be  simulated. 
DAFSScenario  elements  consist  of: 

•  Version:  The  version  of  the  input  file — used  to  determine  the  schema  of 
the  input. 

•  Type:  Currently  only  ‘Attack’  is  used. 

•  optimizelnterval:  Time  between  optimization  events  for  the  CVO. 

•  bdaFactor:  A  number  between  0.0  and  1.0  representing  the  probability 
that  BDA  will  be  correct  (not  used). 

•  replications:  Number  of  replications  to  be  done. 

•  stopTime:  Length  of  the  scenario. 

b.  GCS 

The  Ground  Control  Station  (GCS)  table  indicates  the  name,  the 
associated  LRS,  and  control  capacity  of  each  ground  control  station.  A  GCS  and  its 
associated  LRS  constitute  an  operating  base  for  the  launch,  control,  and  recovery  of  ISR 
platforms.  The  GCS  table  elements  are: 

•  Name:  A  unique  identifier  for  each  GCS. 

•  LRS:  The  name  of  the  LRS  associated  with  the  GCS. 

•  Capacity:  The  maximum  number  of  platforms  (i.e.,  UAVs)  that  the  GCS 
can  control  at  any  one  time.  Only  the  first  entry  for  each  GCS  is  used. 


4  Data  table  names,  letter  case  and  spelling  are  taken  directly  from  the  current  version  of  the  JDAFS 
input  tables. 

^  Descriptions  of  the  elements  in  the  data  tables  are  taken  from  either  the  ASC-U  Users/ Analysts 
Manual  or  the  JDAFS  Users  Manual  as  applicable.  Where  necessary,  the  wording  has  been  changed  to 
match  the  current  usage  in  JDAFS. 
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c.  GCSLocation 

GCS  Location  is  associated  with  the  GCS  table  to  provide  the  geographic 
location  of  the  GCS  and  the  time  at  which  it  becomes  available.  For  this  study  the  GCSs 
are  available  throughout  the  entire  scenario  and  their  location  is  fixed.  The  GCSLocation 
table  is  comprised  of  the  following  elements: 

•  GCS:  Name  of  this  GCS  from  the  GCS  table. 

•  Time:  The  H-hour  at  which  this  GCS  is  at  the  location  designated  by  the 
LocationX  and  LocationY  elements. 

•  LocationX:  The  x  coordinate  of  this  GCS  at  the  time  in  the  Time  element. 

•  LocationY:  The  y  coordinate  of  this  GCS  at  the  time  in  the  Time  element. 

d.  LinearKillProbability 

The  linear  kill  probability  table  provides  the  elements  necessary  to 
evaluate  the  outcome  of  a  firing  event.  Each  munition  type  is  named  and  must  be 
specifically  matched  with  a  target  platfonn  vulnerable  to  attack  with  the  weapon.  The 
minimum  and  maximum  range  for  the  munition  type  and  the  respective  kill  probabilities 
(Pk)  are  also  defined  in  this  table.  Elements  of  the  LinearKillProbability  table  are: 

•  munitionType:  The  munition  for  which  the  kill  probability  is  defined. 
This  must  be  defined  in  a  MunitionType  element. 

•  platformType:  The  target  platfonn  for  the  LinearKillProbability 
definition.  This  must  be  defined  in  a  PlatfonnType  element. 

•  class:  JAVA  class  that  adjudicates  the  firing  action. 

•  minRangePK:  The  probability  of  kill  against  a  target  platfonn  at 
minimum  range. 

•  maxRangePK:  The  probability  of  kill  against  a  target  platfonn  at 
maximum  range. 

•  minRange:  Minimum  range  at  which  a  target  can  be  engaged. 

•  maxRange:  Maximum  range  at  which  a  target  can  be  engaged. 
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e.  LOS 

The  line-of-sight  (LOS)  table  provides  the  necessary  data  for  the  two  stage 
assignment  problem.  The  values  are  pre-calculated  to  determine  the  minimum  altitude  at 
which  there  is  an  unobstructed  line  of  sight  between  a  mission  and  a  mission  (target). 
The  elements  that  make  up  the  LOS  table  are: 

•  Mission:  The  name  of  the  mission  from  the  Mission  table  that  corresponds 
the  Mission  Area  for  which  the  minimum  altitude  to  maintain  line-of-sight 
is  designated  in  the  MinLOSAlt  element. 

•  MissionArea:  The  name  of  the  Mission  Area  from  the  MissionArea  table 
that  corresponds  the  Mission  for  which  the  minimum  altitude  to  maintain 
line-of-sight  is  designated  in  the  MinLOSAlt  element. 

•  MinLOSAlt:  The  minimum  altitude  in  meters  required  to  maintain  line- 
of-sight  between  the  Mission  Area  and  the  Mission  locations. 

f.  LRS 

Launch  and  recovery  sites  (LRS)  are  named  and  the  capacity  is  defined  in 
the  LRS  table.  A  LRS  and  its  associated  GCS  constitute  an  operating  base  for  the  launch, 
control,  and  recovery  of  ISR  platforms.  The  LRS-GCS  association  is  done  in  the  GCS 
table.  The  LRS  elements  consist  of: 

•  Name:  A  unique  identifier  for  each  LRS. 

•  Capacity:  The  maximum  number  of  each  ISR  platform  at  the  LRS. 

g.  LRSLocation 

LRS  Location  is  associated  with  the  LRS  table  to  provide  the  geographic 
location  of  the  LRS  and  the  time  at  which  it  becomes  available.  For  this  study  the  LRSs 
are  available  throughout  the  entire  scenario  and  their  location  is  fixed.  The  LRSLocation 
table  is  comprised  of  the  following  elements: 

•  LRS:  The  name  of  the  LRS  from  the  LRS  table. 

•  Time:  The  FI-hour  at  which  this  LRS  is  at  the  location  designated  by  the 
LocationX  and  LocationY  elements. 

•  LocationX:  The  x  coordinate  of  this  LRS  at  the  time  in  the  Time  element. 

•  LocationY:  The  y  coordinate  of  this  LRS  at  the  time  in  the  Time  element. 
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h.  MetaData 

The  metadata  table  drives  how  a  design  scenario  will  be  run  by  JDAFS. 
Most  significantly,  the  MetaData  table  defines  the  type  of  optimization  algorithm  to  be 
implemented,  the  length  of  the  simulation  scenario,  the  time  between  optimizations,  and 
the  location  of  the  origin  for  the  scenario  coordinate  system.  Elements  in  the  MetaData 
table  are: 

•  InputFormatVersion:  Reports  the  current  version  of  the  input  format. 

•  CVOType:  Identifies  CVO  JAVA  class  to  use. 

•  ScenarioLength  is  the  length  of  the  scenario  to  be  run  in  simulation 
hours. 

•  Optimizationlnterval  is  the  amount  of  time  in  simulation  hours  between 
reallocation  of  the  1SR  platforms. 

•  Reportlnterval:  Frequency  of  report  intervals  (not  used). 

•  DiscountFactor:  Discount  for  time  into  the  future  (not  used). 

•  EarlyReturn:  Heuristic  (not  used). 

•  SecondaryAreas:  Heuristic  (not  used). 

•  AppendAreas:  Heuristic  (not  used). 

•  LocationType:  Heuristic  (not  used). 

•  OriginLat:  Sets  origin  latitude  for  the  scenario. 

•  OriginLng:  Sets  origin  longitude  for  the  scenario. 

•  CostToLaunch:  (not  used). 

i.  Mission 

The  targets  for  collection  by  the  ISR  platforms  are  called  missions.  Each 
mission  is  defined  by  a  name,  a  location,  the  type  of  sensor  required  to  collect  on  the 
mission,  and  the  value  the  correct  sensor  achieves  per  hour  for  coverage  of  the  mission. 
The  Mission  table  elements  consist  of: 

•  Name:  Unique  identifier  for  each  mission  (target). 

•  LocationX:  The  x  coordinate  of  the  mission  (target). 

•  LocationY :  The  y  coordinate  of  the  mission  (target). 

•  StartTime:  The  time  that  the  sensor  requirement  starts,  usually  in  H-hours 
(not  used). 
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•  EndTime:  The  time  that  the  sensor  requirement  ends,  usually  in  H-hours 
(not  used). 

•  RequiredCapability:  The  sensor  type  required  to  detect  this  mission. 

•  ValueRate:  The  rate  at  which  an  ISR  platfonn  with  the  correct  sensor 
receives  credit  when  the  mission  (target)  is  in  the  ISR  platform’s  sensor 
footprint  (all  values  assigned  as  1.0  for  this  study). 

j.  MissionArea 

A  mission  area  is  a  notional  on-station  location  for  the  ISR  platforms 
consisting  of  a  name  and  a  location.  The  elements  in  the  MissionArea  data  table  are: 

•  Name:  Unique  identifier  for  each  mission  area  (patrol  location). 

•  LocationX:  The  x  coordinate  of  the  mission  area  (patrol  location). 

•  LocationY:  The  y  coordinate  of  the  mission  area  (patrol  location). 

k.  MissionPackageLocation 

The  mission  package  location  indicates  how  many  of  the  mission 
packages,  defined  in  the  MissionPackageType  table,  are  located  at  each  LRS  (base).  A 
new  row  is  required  for  each  mission  package  at  a  LRS.  The  MissionPackageLocation 
attributes  are: 


•  LRS:  Unique  identifier  for  the  Launch  and  Recovery  Site  at  which  the 
MissionPackageType  (ISR  platform)  is  located.  Each  LRS  can  only  be 
associated  with  one  MissionPackageType  (ISR  Platfonn).  Multiple  LRSs 
can  be  associated  with  a  single  geographic  location  in  the 
MissionPlatformLocation  table  using  the  UnitName  element. 

•  MissionPackageType:  Name  of  the  ISR  platfonn  associated  with  the 
LRS. 

•  QTY:  Number  of  MissionPacakgeTypes  at  the  LRS. 


/.  MissionPackageType 

The  mission  package  type  defines  the  sensor  packages  that  ISR  platforms 
may  carry.  Each  package  has  a  unique  number  followed  by  unique  names  of  sensors  that 
together  make  up  the  package.  Each  row  of  the  MissionPackageType  table  represents 
one  sensor  package.  The  MissionPackageType  elements  are: 
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•  Name:  Unique  name  of  an  ISR  platfonn. 

•  CapabilityType:  Type  of  sensor  payload  associated  with  the  ISR  platform 
in  the  Name  element.  If  the  ISR  has  more  than  one  sensor  payload,  each 
sensor  must  be  listed  individually  with  the  corresponding  Name  element. 

m.  MissionPlatformLocation 

Each  row  of  the  mission  platform  location  table  defines  the  number  of  ISR 
platforms  of  a  particular  type  assigned  to  a  LRS  (base).  The  MissionPlatformLocation 
elements  are: 

•  LRS:  Unique  identifier  for  the  Launch  and  Recovery  Site  at  which  the 
MissionPackageTypes  (ISR  platform)  are  located.  Each  LRS  can  only  be 
associated  with  one  MissionPackageType  (ISR  Platfonn).  Multiple  LRSs 
can  be  associated  with  a  single  geographic  location  by  using  the 
UnitName  element. 

•  UnitName:  Unique  name  for  a  unit  (base,  airfield,  or  launch  platform) 
which  can  be  associated  with  multiple  LRSs. 

•  PlatformType:  Type  of  ISR  platfonn  at  an  LRS  associated  with  a  specific 
unit  (base,  airfield,  or  launch  platfonn). 

•  Qty:  Number  of  MissionPlatformTypes  at  the  MissionPlatformLocation 
(unit). 

•  StartAvailable:  The  time  that  the  mission  platform  becomes  available, 
usually  in  H-hours. 

•  End  Available:  The  time  that  the  mission  platform  is  no  longer  available, 
usually  in  H-hours. 

n.  MissionPlatformType 

The  mission  platfonn  type  table  contains  all  the  performance 
characteristics  for  the  ISR  platforms.  The  elements  of  the  MissionPlatformType  are: 

•  Name:  The  name  of  an  individual  type  of  ISR  platfonn. 

•  TransitionTime:  The  amount  of  time,  in  hours,  between  the  recovery  of  a 
platform  and  the  time  that  it  is  available  for  a  new  mission. 

•  AirSpeed:  The  rate  of  travel  of  the  ISR  platfonn  in  meters  per  hour. 

•  ControlRadius:  The  maximum  distance  a  platform  requiring  a  Ground 
Control  Station  can  be  from  a  controlling  CGS  for  effective  control 
(Assumed  to  be  via  SATCOM  and  not  used). 
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•  OperationalEnd:  The  maximum  time  a  platform  can  be  away  from  an 
LRS,  the  platform’s  endurance. 

•  TimeHorizon:  The  time  used  for  the  optimization.  Time  horizon  is 
usually  1  to  1.5  times  the  operating  time.  This  value  may  need  to  be 
adjusted  if  transition  time  is  greater  than  half  the  operating  time.  This 
value  is  critical  to  detennining  what  missions  will  be  considered  in  the 
look-ahead  time  window  to  launch  a  UAV  if  GCS  capacity  is  available. 

•  MaxAltitude:  Maximum  operational  altitude  of  the  platfonn  in  meters. 
o.  Mover 

The  mover  table  enables  the  creation  of  different  types  of  platforms.  In 
this  case  only  red  or  threat  platforms  (SAMs)  are  created.  The  mover  platforms  are 
assigned  a  name,  a  type  (i.e.,  SAM),  CVO  type  and  an  affiliation.  Multiple  copies  of 
identical  platforms  can  be  created,  if  required.  Note  that  ISR  platform  creation  is  done  in 
a  separate  process  in  another  table.  The  specific  elements  in  the  Mover  table  are: 

•  name:  An  identifier  given  to  track  agents  within  the  simulation.  If 
multiple  platforms  are  created,  they  are  given  unique  numbers  in  addition 
to  the  name. 

•  type:  The  fully  qualified  name  of  the  implanting  Mover  class.  This  must 
be  specified  in  the  PLATFORM  table  of  the  database. 

•  qty:  The  number  of  identical  platform  that  will  be  created.  Defaults  to  0. 

•  assignment:  specifies  whether  the  platform  will  be  controlled  by  a  fires 
CVO(“fires”),  a  sensor  CVO(“sensor”),  both(“both”)  or  none  (“none”). 

•  affiliation:  “Red”  or  “Blue”  as  listed  in  the  SIDE  table  of  the  database 
(required). 

•  xLoc:  The  initial  x  coordinate  of  the  agent  unless  a  Box  is  used.  If  a  Box 
is  used,  enter  0  as  the  initial  x  coordinate. 

•  yLoc:  The  initial  y  coordinate  of  the  agent  unless  a  Box  is  used.  If  a  Box 
is  used,  enter  0  as  the  initial  y  coordinate. 

•  MaxSpeed:  The  maximum  platform  speed  in  meters  per  hour  (not  used). 

•  OperationalEndurance:  The  maximum  operational  endurance  of  the 
platform  in  hours  (not  used). 

•  formationLeader:  Defines  the  lead  platform  when  movers  travel  in 
formation  (not  used). 
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p.  MoverManager 

The  mover  manager  controls  the  movement  of  mover  platforms  when  they 
are  not  given  assignments  from  a  CVO.  This  table  must  be  completed  for  each  mover 
even  if  none  of  the  movers  defined  will  actually  travel.  In  this  study,  all  of  the  mover 
platforms  are  fixed  and  do  not  move.  The  MoverManager  attributes  are: 

•  ID:  An  automatically  generated  number  to  track  the  different  agents. 

•  class:  The  fully  qualified  name  of  the  MoverManager  class.  The  typical 
values  are: 

•  DAFS.platfonn.DAFSPathMoverManager 

•  DAFS.platfonn.DAFSPatrolMoverManager 

•  DAFS.platfonn.NAIMoverManager 

•  mover:  The  unique  name  given  to  the  agent  from  the  MOVE  table. 

•  delay:  Optional  double  value  (must  be  >=0.0).  Specifies  the  time  after  0.0 
that  the  MoverManager  will  become  active.  If  omitted,  then  an  explicit 
call  to  start()  is  required  to  activate  the  MoverManager. 

•  startOnReset:  Optional  Boolean  (“true”  or  “false”);  if  “true”,  the 
specified  MoverManager  will  commence  its  operation  at  the  start  of  the 
simulation  without  any  explicit  command.  If  “false”  or  omitted,  an 
explicit  call  to  start()  is  required  for  the  MoverManager  to  become  active. 

q.  Munition 

Munition  is  the  definition  for  a  weapon  type  consisting  of  a  unique 
identification  number,  a  noun  name,  and  a  mover  assignment.  The  mover  assignment 
determines  the  firing  platfonn  for  the  weapon.  The  operational  parameters  for  the 
munition  are  characterized  in  the  MunitionType  table.  The  Munition  elements  are: 

•  ID:  Automatically  generated  identification  number  for  the  individual 
munitions. 

•  Type:  Name  of  the  munition  type. 

•  mover:  Name  of  the  mover  (TEL,  prime  mover,  aircraft,  etc.)  upon  which 
the  munition  is  loaded. 
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r.  MunitionType 

The  munition  type  table  provides  the  parameters  for  an  individual 
munition  defined  in  the  Munition  table.  Key  attributes  for  the  munition  type  are  the 
maximum  effective  blast  radius,  the  minimum  and  maximum  effective  ranges,  and  speed. 
The  columns  in  the  MunitionType  table  are: 

•  MUNITION:  A  unique  name  for  the  ordinance. 

•  WEIGHT :  Weight  in  pounds  of  the  weapon  (not  used). 

•  MER:  Effective  blast  radius  in  meters  (minimum  is  1.0). 

•  MINRANGE:  Minimum  range  in  meters. 

•  MAXRANGE:  Maximum  range  in  meters. 

•  LOAD:  The  number  of  rounds  included  in  a  basic  load  out  (initial 
quantity  per  platform). 

•  SPEED:  The  speed  of  the  round  in  meters  per  hour. 

•  ALGORITHM:  Determines  what  algorithm  should  be  used  dependent  on 
type  of  munition. 

•  BURST_SIZE:  For  indirect  fires,  defines  the  effects  radius  from  impact 
(not  used). 

•  SUBMUNITION:  Allows  for  submunitions  and  their  effects  (not  used). 

s.  PlatformType 

All  non-ISR  platform  entities  that  are  not  mover  entities  in  a  scenario  must 
be  defined  in  the  platfonn  type  table.  SAM  is  the  only  PlatformType  used  in  this  study. 
The  PlatformType  elements  are: 

•  name:  Link  to  mover. 

•  value:  Value  used  in  fires  CVO,  set  to  zero  and  not  used  in  this 
experiment. 

t.  Route 

The  route  table  provides  a  method  for  controlling  the  flight  path  of  an  ISR 
platform  during  travel  between  its  home  base  (LRS/GCS)  and  the  assigned  mission  area. 
Each  entry  in  the  table  contains  a  single  enumerated  waypoint.  Multiple  entries  are 
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required  when  more  that  one  waypoint  is  necessary.  If  no  routing  is  defined,  the  aircraft 
travel  in  a  shortest  path  straight  line  between  the  base  and  the  mission  area.  The  Route 
elements  are: 

•  StartLocation:  Name  of  the  LRS  at  which  the  mission  originates. 

•  MissionArea:  Name  of  the  destination  Mission  Area  for  the  route. 

•  PointNumber:  Number  indicting  the  sequence  of  the  waypoint  in  the 
route. 

•  LocationX:  The  x  coordinate  of  the  waypoint. 

•  LocationY :  The  y  coordinate  of  the  waypoint. 

u.  Sensor 

The  sensor  table  creates  a  relationship  between  a  mover  and  a  sensor  type 
for  the  sensors  defined  in  the  SensorType  table.  There  is  no  relationship  between  the 
Sensor  table  and  SensorRange  table.  The  types  of  sensor  relationships  created  in  this 
study  are  strictly  for  red  or  threat  platforms.  The  Sensor  elements  are: 

•  ID:  AutoN umber  used  to  track  the  sensors. 

•  type:  As  described  under  the  SensorType  Table. 

•  mover:  The  agent  the  sensor  is  assigned  to. 

v.  SensorRange 

Sensor  range  provides  the  maximum  sensor  range  for  the  sensors  to  be 
carried  by  the  ISR  platforms.  The  types  of  sensors  defined  in  this  table  consist  of  three 
elements,  the  platform  type  to  carry  the  sensor,  the  sensor  capability  type,  and  the  sensor 
range.  This  allows  the  creation  of  sensors  of  similar  types  with  different  ranges.  For 
example,  the  EO  sensor  carried  by  a  U-2  can  a  have  a  different  range  than  an  EO  sensor 
mounted  on  a  RQ-1.  There  is  no  relationship  between  the  SensorRange  table  and  Sensor 
or  Sensor  type  tables.  The  sensors  created  in  this  table  are  strictly  for  blue  platfonns. 
Columns  in  the  SensorRange  table  consist  of: 

•  MissionPlatformType:  Name  of  the  platfonn  that  carries  the  sensor. 

•  MissionCapabilityType:  Type  of  coverage  provided  by  the  sensor.  The 
three  capability  types  defined  are  EO/IR,  SAR,  and  SIGINT. 

•  Range:  Maximum  range  of  the  sensor  in  meters. 
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w.  SensorType 

Sensor  type  creates  a  name  and  range  for  sensors  that  can  be  mounted  on 
mover  platforms.  For  this  study,  the  SensorType  table  is  used  to  create  notional  SAM 
fire  control  radars.  There  is  no  relationship  between  the  SensorType  table  and 
SensorRange  table.  The  SensorType  elements  are: 

•  name:  A  unique  name  for  the  sensor. 

•  maxRange:  The  maximum  range  of  the  sensor  in  meters. 

•  class:  The  fully  qualified  class  name  for  the  sensor. 

x.  Side 

Side  detennines  the  affiliation  of  the  entities  in  the  simulation  (i.e.,  red  or 
blue).  The  side  table  contains  only  one  element: 

•  name:  Designates  affiliation.  Red  indicates  enemy  and  blue  indicates 

own  forces. 

y.  SimEntity 

The  SimEntity  element  is  where  platforms  are  defined.  Each  platform  is 
defined  in  a  Mover  sub-element. 

•  elements:  The  SimEntity  has  no  attributes. 

3.  Execution 

Simulation  runs  in  JDAFS  can  be  executed  in  a  number  of  ways.  Single  runs, 
including  multiple  replications  of  the  same  scenario,  can  be  made  using  either  the  JDAFS 
Graphical  User  Interface  (GUI)  (See  Figure  6),  or  the  command  line.  The 
implementation  of  more  complex  experiments  requiring  multiple  replications  of  many 
design  points  should  be  run  from  a  computing  cluster. 
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Figure  6.  JDAFS  Graphical  User  Interface 

4.  Output 

Upon  completion  of  each  simulation,  output  tables  are  generated  or  populated 
depending  on  the  method  used  to  run  the  scenario.  Single  runs  from  the  JDAFS  GUI 
produce  new  Microsoft  Access  databases  that  must  be  manually  saved  following  the 
completion  of  the  simulation.  Output  from  command  line  or  computing  cluster  runs  are 
saved  back  to  output  data  tables  in  the  original  input  database. 

JDAFS  has  the  ability  to  create  multiple  output  tables  to  capture  MOE  and  model 
run  parameters.  These  output  tables  include:  Acquisition,  Coverage,  Coverage  by  Type, 
Coverage  Delay,  Killer-Victim  Scoreboard,  Mission  Assignment,  Run  Information,  and 
Schedule.  Only  the  output  tables  capturing  the  MOEs  of  interest  (Coverage,  Coverage  by 
Type,  and  Killer-Victim  Scoreboard)  for  this  research  will  be  described  in  detail.  For  an 
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explanation  of  the  other  tables,  see  the  ASC-U  Users/Analyst  Manual  (Ahner  et  al. 
2006b)  The  following  data  tables  and  their  associated  elements  are  created  at  the 
completion  of  each  run:6’7 

a.  Coverage 

The  Coverage  table  contains  one  row  for  each  Mission  in  the  scenario. 
The  elements  in  the  Coverage  table  are: 

•  Replication:  Number  indicating  the  simulation  run  which  provided  the 
data. 

•  Mission:  The  name  of  the  Mission.  For  this  study,  the  mission  name  is  a 
numerical  code  where  the  number  before  the  decimal  represents  the 
mission  name  and  the  number  after  the  decimal  identifies  the  type  of 
sensor  required  to  cover  the  mission. 

•  Open  Time:  The  total  time,  in  hours,  that  the  mission  was  vulnerable  to 
coverage  by  the  required  sensor. 

•  Covered  Time:  The  total  amount  of  time,  in  hours  that  the  mission  was 
covered  by  the  required  sensor. 

•  Coverage:  The  percentage  of  time  that  the  mission  was  covered  by  at  least 
one  sensor  during  the  replication. 

•  Value  Rate:  The  value  received  per  unit  time  by  the  covering  platfonn. 

•  Total  Value:  The  total  value  obtained  from  this  Mission  during  the 
scenario. 

b.  Coverage  by  Type 

The  Coverage  by  Type  table  contains  one  row  for  each  Mission/ISR 
platform  type.  The  following  columns  make  up  the  Coverage  by  Type  table: 

•  Replication:  Number  indicating  the  simulation  run  which  provided  the 
data. 

•  Mission:  The  name  of  the  Mission.  For  this  study,  the  mission  name  is  a 
numerical  code  where  the  number  before  the  decimal  represents  the 
mission  name  and  the  number  after  the  decimal  identifies  the  type  of 
sensor  required  to  cover  the  mission. 


6  Data  table  names,  letter  case  and  spelling  are  taken  directly  from  the  current  version  of  the  JDAFS 
input  tables. 

7  Descriptions  of  the  elements  in  the  data  tables  are  taken  from  the  ASC-U  Users/Analysts  Manual. 
Where  necessary,  the  wording  has  been  updated  to  match  the  current  usage  in  JDAFS. 
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•  Mission  Platform  Type:  The  type  of  platform  assigned  to  the  Mission. 

•  Open  Time:  The  total  time,  in  hours,  that  the  mission  was  vulnerable  to 
coverage  by  the  required  sensor. 

•  Covered  Time:  The  total  amount  of  time,  in  hours  that  the  mission  was 
covered  by  the  required  sensor. 

•  Coverage:  The  percentage  of  time  that  the  mission  was  covered  by  at  least 
one  sensor  during  the  replication. 

•  Value  Rate:  The  value  received  per  unit  time  by  the  covering  platform. 

•  Total  Value:  The  total  value  obtained  from  this  Mission  during  the 
scenario. 


c.  KillerVictimS coreboard 

The  Killer-Victim  Scoreboard  provides  a  means  of  measuring  attrition  for 
a  scenario.  Each  row  in  the  table  gives  details  on  individual  engagements.  The  Killer- 
Victim  Scoreboard  is  composed  of  the  following  columns: 

•  Replication:  Number  indicating  the  simulation  run  that  provided  the  data. 

•  Sim  Time:  Time  in  the  simulation  that  the  firing  event  occurred. 

•  Firing  Platform:  Name  of  the  platform  that  launched  the  weapon. 

•  Munition:  Type  of  munition  fired  by  the  platform. 

•  launch  x:  The  x  coordinate  of  the  firing  platform  at  the  time  of  the  firing 
event. 

•  launch  y:  The  y  coordinate  of  the  firing  platfonn  at  the  time  of  the  firing 
event. 

•  weight:  Weight  of  the  munition  (not  used). 

•  Target:  The  unique  name  of  the  platform  targeted. 

•  target  x:  The  x  coordinate  of  the  target  at  the  time  of  the  firing  event. 

•  target  y:  The  y  coordinate  of  the  target  at  the  time  of  the  firing  event. 

•  type:  Type  of  platform  targeted. 

•  target  value:  Value  of  the  target  (not  used). 

•  target  side:  Affiliation  of  the  target  platfonn. 

•  outcome:  Result  of  the  engagement,  missed  or  killed. 
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III.  SCENARIO  DEVELOPMENT 


A.  SCENARIOS 

The  scenarios8  built  for  this  study  are  designed  to  conduct  a  trade-off  analysis  for 
Joint  ISR  operations.  Two  types  of  scenarios  were  generated  for  examination  and 
comparison  within  JDAFS.  The  first  scenario  is  a  non-penetrating  scenario  (See  Figures 
7  and  8,  p.  35)  where  the  ISR  platforms  do  not  penetrate  the  Country  of  Interest’s  (COI) 
national  airspace.  The  internationally  accepted  buffer  of  22  kilometers  is  respected  on  all 
flights  and  waypoints  have  been  implemented  to  prevent  ingressing  and  egressing  aircraft 
from  violating  the  COI’s  sovereign  airspace.  The  second  scenario  (See  Figures  9  and  10, 
p.  36)  assumes  that  conditions  have  changed  to  allow  the  violation  of  the  COI’s  airspace. 
With  the  incursions  into  the  COI’s  territory  comes  the  risk  of  engagement  by  air  defense 
assets,  in  this  case  surface-to-air  missiles  (SAMs). 

The  country  of  interest  for  the  scenario  is  depicted  as  a  hexagon  measuring 
1000km  between  any  of  its  widest  points.  The  size  of  the  country  allows  the  missions 
(ISR  targets)  to  be  widely  dispersed  and  ensures  that  even  the  most  capable  sensors 
included  in  the  study  do  not  cover  excessive  portions  of  the  country  from  a  single  mission 
area.  The  missions  are  dispersed  non-unifonnly  throughout  the  country  to  achieve  a 
sense  of  realism.  While  the  mission  dispersal  is  not  unifonn,  it  is  also  not  random.  Care 
is  taken  to  place  the  missions  such  that  the  various  mission  sensor  requirements  are 
widely  distributed  to  limit  disproportionate  coverage  by  a  single  platfonn.  Mission  areas 
were  initially  designed  to  ring  the  entire  COI;  however,  allowing  complete  access  to  all 
countries  surrounding  a  COI  may  be  unrealistic.  Overflight  of  mission  areas  along  the 
western  border  and  a  portion  of  the  southern  border  were  deemed  to  be  denied.  Four 
operating  bases  for  the  ISR  platforms  were  placed  at  varying  distances  in  the  non-denied 
areas  surrounding  the  COI  to  allow  for  different  transit  times  from  base  to  mission  area. 
The  SAM  sites  were  distributed  across  the  COI  with  the  strategic  SAMs  (SA-2  and  SA- 


8  There  is  no  intention  to  depict  any  real  world  operational  area  or  threat  order  of  battle. 
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10)  treated  as  missions.  The  tactical  SAMs  (SA-6)  were  placed  as  point  defense  weapons 
at  three  separate  mission  types.  See  Figures  7,  8,  9,  and  10  (pp.  35  and  36)  for  the 
specific  placement  of  entities  in  this  scenario. 

The  design  of  a  scenario  for  use  in  exercising  the  JDAFS  model  requires  the 
creation  of  the  following  components: 

1.  Missions  (Targets) 

For  the  purposes  of  this  simulation,  25  missions  or  targets  of  varying  types  were 
created  and  populated  throughout  the  COI  (See  Figures  7,  8,  9  and  10,  pages  35  and  36). 
Target  types  consist  of  command  and  control  (C2)  nodes,  surface-to-air  missiles  (SAMs), 
short  range  ballistic  missiles  (SRBMs),  medium  range  ballistic  missiles  (MRBM),  long 
range  ballistic  missiles  (LRBMs),  airfields,  weapons  of  mass  destruction  facilities  (WMD 
FACs),  military  facilities  (MIL  FACs),  and  ammunition  storage  (AMMO  STOR) 
facilities.  Intelligence  information  for  each  type  of  target  can  only  be  collected  against 
by  specific  types  of  sensors.  The  mission/sensor  interactions  must  be  explicitly  created  in 
the  input  tables  and  result  in  the  creation  of  55  target  elements  from  the  25  actual 
missions  (targets).  The  missions  and  their  sensor  requirements  are  listed  in  Table  1. 
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MISSION 

TYPE 

EO/IR 

SAR 

SIGINT 

1 

C2 

X 

2 

SA-2 

X 

X 

X 

3 

WMD  FAC 

X 

4 

IRBM 

X 

X 

X 

5 

AIRFIELD 

X 

X 

X 

6 

SRBM 

X 

X 

X 

7 

C2 

X 

8 

C2 

X 

9 

SA-2 

X 

X 

X 

10 

SA-2 

X 

X 

X 

11 

C2 

X 

12 

AMMO  STOR 

X 

X 

13 

AMMO  STOR 

X 

X 

14 

IRBM 

X 

X 

X 

15 

LRBM 

X 

X 

X 

16 

SRBM 

X 

X 

X 

17 

MIL  FAC 

X 

X 

18 

WMD  FAC 

X 

19 

MIL  FAC 

X 

X 

20 

MIL  FAC 

X 

X 

21 

IRBM 

X 

X 

X 

22 

MIL  FAC 

X 

X 

23 

C2 

X 

24 

AIRFIELD 

X 

X 

X 

25 

SA-10 

X 

X 

X 

Table  1.  Targets  and  Required  Sensors  for  Collection 

2.  Mission  Areas 

Mission  areas  (See  Figures  7,  8,  9  and  10,  pp.  35  and  36)  are  the  locations  created 
to  simulate  on-stations  positions  for  the  ISR  platforms.  The  non-penetrating  scenario  has 
14  mission  areas  positioned  outside  the  COI  and  the  penetrating  adds  an  additional  12 
mission  areas  inside  the  COT  The  14  non-penetrating  mission  areas  are  what  remain  of 
an  originally  plotted  24  mission  areas  after  deleting  the  points  deemed  to  be  inaccessible 
due  to  being  in  a  country  that  has  denied  overflight  rights.  For  the  placement  of  the  12 
additional  penetrating  mission  areas,  no  regard  was  given  to  minimum  risk  flight  routing 
in  order  to  enable  the  examination  of  the  effects  of  attrition. 
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3. 


Bases 


Four  airfields  or  bases  (See  Figures  7,  8,  9  and  10,  pp.  35  and  36)  were  created  by 
assigning  fixed  common  sets  of  coordinates  to  GCSs  and  LRSs.  ISR  platforms  are  then 
associated  with  specific  GCSs  and  LRSs  to  establish  the  asset  lay  down  at  each  base. 
The  four  bases  are  varying  distances  from  the  COI  to  take  into  account  the  differences  in 
transit  times  that  would  be  expected  in  real  world  operations. 

4.  Threats 

The  threats  depicted  in  this  scenario  (See  Figures  7,  8,  9  and  10,  pp.  35  and  36) 
are  all  surface-to-air  missiles  (SAMs).  Three  different  SAM  systems  of  varying  ranges 
were  selected  for  inclusion  in  the  scenario:  the  SA-2,  SA-6,  and  SA-10.  The  Pk  for  the 
SAMs  was  assumed  to  be  the  same  against  all  ISR  platforms.  Studies  conducted  with 
real  world  operational  data  and  parameters  would  have  a  separate  Pk  for  each  individual 
ISR  platform.  The  SA-6,  a  mobile  platfonn,  is  used  in  fixed  positions  for  this  scenario. 
The  SA-6s  are  not  treated  as  missions  or  targets  for  this  study,  but  are  all  collocated  with 
missions. 

The  operational  parameters  for  the  threat  platforms  used  in  JDAFS  for  this  thesis 
are  listed  in  Table  2.  For  real  world  studies  conducted  in  a  classified  environment, 
operational  parameters  would  be  provided  by  the  organization  sponsoring  the  research. 


THREAT 

MIN  RANGE 
(meters) 

MAX  RANGE 
(meters) 

MIN  RANGE  Pk 

MAX  RANGE  Pk 

SA-2 

1000 

50000 

0.6 

0.5 

SA-6 

1000 

24000 

0.6 

0.5 

SA-10 

1000 

200000 

0.6 

0.5 

able  2.  Surface-to-Air  Missile  Parameters 


While  not  used  in  this  scenario,  other  types  of  threat  platfonns  can  also  be 
depicted  in  JDAFS  such  as,  Anti-Aircraft  Artillery  (AAA)  and  fighter/interceptor  aircraft. 
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Figure  7.  JDAFS  Non-Penetrating  Scenario  as  Designed  {Best  viewed  in  color ) 


» 

■* 

►  4 

aO  C 

z 

O  L 

<kA 

>  oAo 

A  o 

*  A  O 

A  O  -O 

A 

A  J 

* 

A  a$T 

A  *0 

O  Aq 

'i  n  o 

►4 

Figure  8.  JDAFS  Non-Penetrating  Scenario  in  JDAFS  GUI  {Best  viewed  in  color) 
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Figure  9.  JDAFS  Penetrating  Scenario  as  Designed  {Best  viewed  in  color ) 
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Figure  10.  JDAFS  Penetrating  Scenario  in  JDAFS  GUI  {Best  viewed  in  color ) 
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B. 


PLATFORMS 


The  ISR  platforms  selected  for  this  study  are  intended  to  represent  a  broad  sample 
of  theater  and  national  level  capabilities.  It  is  important  to  note  that  the  airframes  used  in 
this  simulation  are  proxies  for  dissimilar  ISR  platforms  to  provide  a  sense  of  realism. 
The  platforms  are  in  no  way,  implicitly  or  explicitly,  intended  to  reflect  real  world 
classified  operational  capabilities. 


1.  RQ-1  Predator 


Figure  1 1 .  RQ- 1  Predator  (From  Global  Aircraft  2007) 

The  RQ-1  Predator  is  a  medium  altitude  and  endurance  unmanned  aerial  vehicle 


that  carries  EO/IR  and  SAR  sensors. 

2.  RQ-4  Global  Hawk 


Figure  12.  RQ-4  Global  Hawk  (From  Tech-Writer.net  2007,  accessed  07  May  2007) 

The  RQ-4  Global  Hawk  is  a  high  altitude  long  endurance  unmanned  aerial  vehicle 
that  carries  EO/IR,  SAR  and  ELINT  sensor  payload  packages. 
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3. 


P-3C  Orion 


Figure  13.  P-3C  Orion  (From  Aerospaceweb.org  2007,  accessed  07  May  2007) 

The  P-3C  (AIP)  Orion  is  a  manned  all-weather  day-night  long  endurance 
maritime  patrol  aircraft  that  carries  EO/IR,  SAR  and  ELINT  sensor  payload  packages. 

4.  RC-135  Rivet  Joint 


Figure  14.  RC-135  Rivet  Joint  (From  Aerospaceweb.org  2007,  accessed  07  May 

2007) 
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The  RC-135  Rivet  Joint  is  a  manned  all-weather  day-night  Signals  Intelligence 
(SIGNINT)  collection  platfonn. 

5.  U-2  Dragon  Lady 


Figure  15.  U-2  Dragon  Lady  (From  Aerospaceweb.org  2007,  accessed  07  May  2007) 


The  U-2  is  a  manned  high  altitude  reconnaissance  aircraft  with  EO/IR,  SAR,  and 
ELINT  packages. 

The  operational  parameters  used  in  JDAFS  for  this  thesis  are  listed  in  Table  3. 9 
For  real  world  studies  conducted  in  a  classified  environment,  operational  parameters 
would  be  provided  by  the  organization  sponsoring  the  research. 


PLATFORM 

MAX 

ALTITUDE 

(meters) 

SPEED 

(meters/hour) 

ENDURANCE 

(hours) 

TRANSITION 

TIME 

(hours) 

RQ-1 

8000 

148160 

24 

1.5 

RQ-4 

18500 

629860 

36 

1.5 

P-3 

9000 

422256 

12 

1.5 

RC-135 

10700 

805629 

11 

1.5 

U-2 

21500 

824140 

10 

1.5 

Table  3.  ISR  Platfonn  Performance  Parameters 


9  The  operational  parameters  for  the  aircraft  in  this  study  are  approximations  based  on  a  variety  of 
open-source  references. 
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C.  SENSORS 

The  sensors  used  in  this  study  are  implemented  as  cookie-cutter  sensors  with  a 
defined  footprint  and  probability  of  detection  of  1.0  (i.e.,  if  a  target  falls  with  in  a  sensors 
footprint,  it  will  be  detected).  Other  sensor  capabilities  may  be  available  in  future 
versions  of  JDAFS. 

1.  Electro-Optical/Infrared  (EO/IR) 

Electro-optical  and  infrared  sensors  are  combined  for  the  purposes  of  this  research 
due  to  their  similarity  in  detection  ranges.  EO  sensors  are  day  only  sensors  that  produce 
images  based  on  visible  light,  similar  to  standard  photographs.  IR  sensors  are  day/night 
sensors  that  produce  images  based  on  thermal  differences.  IR  sensors  typically  perfonn 
better  at  night  when  the  surrounding  environment  tends  to  be  cooler  than  the  target  of 
interest. 


EO  IMAGE  IR  IMAGE 


Figure  16.  EO  and  IR  Images  over  Bosnia  (After  Global  Defense  Review  1997) 

2.  SAR 

The  Synthetic  Aperture  Radar  (SAR)  sensor  is  an  all  weather  day/night  sensor 
that  uses  reflected  radar  energy  to  create  an  image  of  an  area  or  items  of  interest  on  the 
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ground.  Due  to  the  characteristics  of  SAR,  this  sensor’s  actual  performance  is  more 
closely  approximated  by  the  cookie-cutter  sensor  implementation  than  either  the  EO/IR 
or  SIGINT  sensors. 


Figure  17.  SAR  Image  of  the  Pentagon  (After  Sandia  National  Laboratories  2006) 

3.  SIGINT 

Signals  Intelligence  (SIGINT)  sensors  monitor  the  electromagnetic  spectrum  for 
signals  of  interest  based  on  the  ISR  platform’s  sensor  payload  abilities  and  mission 
tasking.  The  SIGINT  sensors  for  this  study  do  not  differentiate  between  communications 
intelligence  (COMINT)  and  electronic  signals  intelligence  (ELINT)  payloads  nor 
whether  the  target  is  emitting,  only  whether  the  target  can  be  collected  against  based  on 
the  footprint  of  the  cookie-cutter  sensor. 

4.  Sensor  Ranges 

The  sensor  ranges  used  in  JDAFS  for  this  thesis  are  listed  in  Table  4  and 
displayed  graphically  in  Figure  18.  For  real  world  studies  conducted  in  a  classified 
environment,  the  sensor  ranges  and  detection  parameters  would  be  provided  by  the 
organization  sponsoring  the  research. 
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PLATFORM 

SENSOR 

RANGE  (meters) 

P-3 

EO/IR 

75000 

P-3 

SAR 

200000 

RC-135 

SIGINT 

300000 

RQ-1 

EO/IR 

50000 

RQ-1 

SAR 

200000 

RQ-4 

EO/IR 

75000 

RQ-4 

SAR 

200000 

RQ-4 

SIGINT 

300000 

U-2 

EO/IR 

100000 

U-2 

SAR 

250000 

U-2 

SIGINT 

400000 

Table  4.  ISR  Platform  Sensor  Ranges 


Figure  18.  Cookie-Cutter  Sensor  Footprints 

D.  CONSTRAINTS,  LIMITATIONS,  AND  ASSUMPTIONS 

The  use  of  modeling  and  simulation  inherently  involves  constraints,  limitations, 
and  assumptions.  As  defined  in  the  ASC-U  User’s  Manual,  “Constraints  are  conditions 
imposed  upon  the  development  that  are  not  under  the  control  of  the  developers. 
Limitations  are  self-imposed  and  are  due  to  choices  that  the  developers  of  ASC-U  made 
to  limit  the  scope  of  the  problem.  Assumptions  are  those  choices  made  to  simplify  the 
problem  for  ease  of  solution”  (Ahner  et  al.  2006c,  9). 

1.  Constraints 

•  JDAFS  improvements  are  limited  to  those  achievable  in  a  three  month 
development  window. 

•  This  study  does  not  consider  cueing  from  other  assets  that  indicate  a  need 
for  greater  fidelity  in  the  resolution  on  ISR  information  required. 

•  No  space  or  ground  based  assets  are  included  in  this  study. 
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Limitations 


Performance  data  is  not  readily  available  due  to  a  lack  of 
performance/capability  databases  and  classification  of  most  relevant  data 
beyond  the  UNCLASSIFIED  level. 

No  consideration  is  made  of  weather  or  environmental  effects  on  sensor 
performance. 

LOS  is  based  on  visual  horizon  calculations  only,  no  terrain  features  are 
modeled. 

The  number  and  location  of  targets  is  fixed. 

Collection  platforms  operate  only  at  fixed  maximum  platform  altitudes. 

Collection  platforms  operate  at  fixed  speeds,  there  is  no  difference 
between  patrol  and  ingress/egress  speeds. 

The  model  currently  has  a  limited  number  of  entities  that  can  be  included 
in  the  simulation  without  overwhelming  the  LpSolver  routine.  The 
maximum  number  of  entities  varies  based  on  the  type  of  scenario  and  the 
optimization  interval. 

Platforms  can  not  detect  targets  during  ingress  or  egress. 

Platforms  cannot  travel  from  mission  area  to  mission  area. 

Assumptions 

The  main  factor  in  determining  success  of  sensors  is  line-of-sight  to  target. 
The  effects  of  weather  and  enviromnental  factors  are  neglected.  These  do 
not  significantly  effect  SAR  sensors,  but  can  significantly  impact  EO/IR 
sensors. 

CONOPS  of  airborne  platfonns  is  at  maximum  altitude  perpendicular  to 
mission  area  location. 

No  camouflage,  concealment,  or  deception  is  employed. 

At  least  two  operating  bases  will  be  available. 

Platforms  penetrating  adversary  national  air  space  will  be  attacked. 
Platforms  outside  adversary  national  air  space  cannot  be  attacked. 

Platform  speeds  are  constant. 

Cycle  time  between  missions  is  constant. 

Platforms  do  not  have  emergent  maintenance  problems  resulting  in  lost 
sorties. 

The  number  of  targets  is  constant,  there  are  no  pop-up  targets. 
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•  There  are  no  mobile  targets. 

•  All  UAV  control  and  datalink  requirements  are  met. 

•  Atmospheric  conditions  do  not  affect  collection,  control,  or  transmission. 
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IV.  MOE,  DOE,  AND  RUN  EXECUTION 


A.  MEASURES  OF  EFFECTIVENESS  (MOE) 

Three  primary  MOEs  are  of  interest  for  analysis  in  this  experiment:  Coverage, 
Coverage  by  Type,  and  Attrition.  The  attrition  MOE  is  only  applicable  to  the  penetrating 
scenario  since,  by  design,  the  ISR  platforms  cannot  be  attacked  if  they  do  not  violate  the 
COI’s  airspace.  JDAFS  records  numerous  other  data  elements  that  could  contribute  to 
the  analysis  of  other  MOEs,  as  required. 

1.  Coverage 

Coverage  is  the  amount  of  time  each  Mission  is  covered  by  an  ISR  platform.  In 
support  of  this  MOE,  JDAFS  records  the  following  (See  Figure  19): 

•  Replication:  Number  indicating  the  simulation  run  which  provided  the 
data. 

•  Mission:  The  name  of  the  Mission.  For  this  study,  the  mission  name  is  a 
numerical  code  where  the  number  before  the  decimal  represents  the 
mission  name  (See  Table  1)  and  the  number  after  the  decimal  identifies 
the  type  of  sensor  required  to  cover  the  mission  (x.  1  =  EO/IR,  x.2  =  SAR, 
x.3  =  SIGINT). 

•  Open  Time:  The  total  time,  in  hours,  that  the  mission  was  vulnerable  to 
coverage  by  the  required  sensor. 

•  Covered  Time:  The  total  amount  of  time,  in  hours,  that  the  mission  was 
covered  by  the  required  sensor. 

•  Coverage:  The  percentage  of  time  that  the  missions  were  covered  by  at 
least  one  sensor  during  the  replication. 

•  Value  Rate:  The  value  received  per  unit  time  by  the  covering  platfonn. 

•  Total  Value:  The  total  value  obtained  from  this  Mission  during  the 
scenario. 


replication  Mission 

openTime 

coveredTime 

coverage 

valueRate 

totalValue 

1  1.3 

240 

220.22436 

0.917601 

1 

220.2244 

1  2.1 

240 

67.872923 

0.282804 

1 

67.87292 

1  2.2 

240 

68.5271731 

0.28553 

1 

68.52717 

1  2.3 

240 

227.174711 

0.946561 

1 

227.1747 

1  3.1 

240 

0 

0 

1 

0 

1  4.1 

240 

220.782606 

0.919928 

1 

220.7826 

Figure  19.  Coverage  Data  Table  Example 
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2.  Coverage  by  Type 

Coverage  by  Type  is  the  amount  of  time  each  Mission  is  cover  by  each  type  of 
ISR  platform.  In  support  of  this  MOE,  JDAFS  records  the  following  (See  Figure  20): 

•  Replication:  Number  indicating  the  simulation  run  which  provided  the 
data. 

•  Mission:  The  name  of  the  Mission.  For  this  study,  the  mission  name  is  a 
numerical  code  where  the  number  before  the  decimal  represents  the 
mission  name  (See  Table  1)  and  the  number  after  the  decimal  identifies 
the  type  of  sensor  required  to  cover  the  mission  (x.  1  =  EO/IR,  x.2  =  SAR, 
x.3  =  SIGINT). 

•  Mission  Platform  Type:  The  type  of  platform  assigned  to  the  Mission. 

•  Open  Time:  The  total  time,  in  hours,  that  the  mission  was  vulnerable  to 
coverage  by  the  required  sensor. 

•  Covered  Time:  The  total  amount  of  time,  in  hours  that  the  mission  was 
covered  by  the  required  sensor. 

•  Coverage:  The  percentage  of  time  that  the  mission  was  covered  by  at  least 
one  sensor  during  the  replication. 

•  Value  Rate:  The  value  received  per  unit  time  by  the  covering  platform. 

•  Total  Value:  The  total  value  obtained  from  this  Mission  during  the 
scenario. 


replication 

Mission 

MissionPlatformType 

openTime 

coveredTime 

coverage 

valueRate 

totalValue 

1 

1.3  P-3 

240 

0 

0 

1 

0 

1 

1.3  RC-135 

240 

19.21385271 

0.08005772 

1 

19.21385271 

1 

1.3  RQ-1 

240 

0 

0 

1 

0 

1 

1.3  RQ-4 

240 

213.5448028 

0.889770012 

1 

213.5448028 

1 

1.3  U-2 

240 

19.0502298 

0.079375957 

1 

19.0502298 

1 

2.1 

P-3 

240 

48.82269319 

0.203427888 

1 

48.82269319 

Figure  20.  Coverage  by  Type  Data  Table  Example 

3.  Attrition 

Attrition  represents  the  number  and  type  of  ISR  platforms  lost.  In  support  of  this 
MOE,  JDAFS  records  the  following  in  the  KillerVictimScoreboard  table  (See  Figure  21): 

•  Replication:  Number  indicating  the  simulation  run  that  provided  the  data. 

•  Sim  Time:  Time  in  the  simulation  that  the  firing  event  occurred. 

•  Firing  Platform:  Name  of  the  platform  that  launched  the  weapon. 

•  Munition:  Type  of  munition  fired  by  the  platform. 
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launch  x:  The  x  coordinate  of  the  firing  platform  at  the  time  of  the  firing 
event. 


launch  y:  The  y  coordinate  of  the  tiring  platfonn  at  the  time  of  the  firing 
event. 

weight:  Weight  of  the  munition  (not  used). 

Target:  The  unique  name  of  the  platform  targeted. 

target  x:  The  x  coordinate  of  the  target  at  the  time  of  the  firing  event. 

target  y:  The  y  coordinate  of  the  target  at  the  time  of  the  firing  event. 

type:  Type  of  platform  targeted. 

target  value:  Value  of  the  target  (not  used). 

target  side:  Affiliation  of  the  target  platfonn. 

outcome:  Result  of  the  engagement,  missed  or  killed. 


Replication!  SimTime  | Firing  Platform 

Munition 

launch  x 

launch  y| 

weight 

Target 

target  x 

target  y  |  type 

target  value|  target 

side|  outcome 

1  0.826690884  SA-101 

SA-10 

0 

0 

1 

U-2_1 

0 

168690.975  U-2 

0  Blue 

killed 

1  2.523543282  SA-6_37 

SA-6 

50000 

-350000 

1 

RQ-1_13 

72226.3106 

-348622.3541  RQ-1 

0  Blue 

killed 

1  2.718760643  SA-6_37 

SA-6 

50000 

-350000 

1 

U-2_9 

70792.75866 

-344680.0863  U-2 

0  Blue 

missed 

1  2.945224124  SA-101 

SA-10 

0 

0 

1 

U-2_9 

7010.649004 

-169279.2848  U-2 

0  Blue 

killed 

1  4.142868882  SA-2_12 

SA-2 

-250000 

250000 

1 

RQ-1_1 

-261357.3946 

294615.5364  RQ-1 

0  Blue 

killed 

1  8.206269191  SA-101 

SA-10 

0 

0 

1 

RQ-1_14 

74908.41726 

-168360.7734  RQ-1 

0  Blue 

missed 

Figure  2 1 .  Killer  Victim  Scoreboard  Example 

B.  DESIGN  OF  EXPERIMENT 

A  design  of  experiments  is  a  structured  scientific  approach  to  determining  how 
various  input  parameters,  or  factors,  impact  the  results  of  an  experiment.  In  order  to  fully 
explore  a  model’s  output,  a  plan  must  be  in  place  to  ensure  the  collection  of  data 
representative  of  the  model’s  entire  response  surface.  In  computer  simulation 
experiments,  the  understanding  of  model  behavior  and  response  is  often  best  done 
through  the  use  of  data  farming. 


The  concept  of  data  farming  originated  in  1998  and  was  further  developed  by 
Marine  Corps  Warfighting  Laboratory’s  Project  Albert.  Data  farming  is  built  on 
“distributed  and  high-performance  computing;  agent-based  simulations  and  rapid  model 
development;  knowledge  discovery  methods;  high  dimensional  data  visualization 
techniques;  design-of-experiments  methods,  human-computer  interfaces;  team  work  and 
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collaborative  environments;  and  heuristic  search  techniques”  (Home  and  Meyer  2004, 
807-808).  This  collection  of  techniques  and  enabling  technologies  allows  a 
comprehensive  examination  of  the  model. 

JDAFS  is  not  currently  configured  to  take  full  advantage  of  the  data  fanning 
concept,  but  is  likely  to  include  future  updates  to  integrate  data  farming  features.  For  this 
study,  the  design  of  experiments,  distributed  computing,  post-processing  of  output,  and 
analysis  all  occur  in  a  piecemeal  manner. 

1.  Factors 

Given  a  database  input  format  and  instance  (scenario),  any  possible  data  element 
in  the  database  is  a  potential  design  factor.  Practically  speaking,  however,  for  a  model 
such  as  JDAFS,  large  groups  of  data  elements  together  can  constitute  a  single  factor.  The 
factors  to  be  varied  in  this  study  are  the  number  of  each  type  of  aircraft  assigned  to  each 
base  and  the  optimization  interval.  Of  the  21  total  factors,  there  are  12  integer  factors 
with  4  levels,  8  integer  factors  with  7  levels,  and  1  continuous  factor. 

a.  Assets  Assigned  to  Bases 

The  number  of  each  type  of  aircraft  assigned  to  each  base  make  up  the 
majority  of  factors  for  this  experiment.  There  are  4  bases  (A,  B,  C,  and  D)  and  5  types  of 
aircraft  (RQ-1,  RQ-4,  P-3,  RC-135,  and  U-2),  for  a  total  of  20  ‘Assets  Assigned  to  Bases’ 
factors.  This  factor  type  is  exclusively  integer  since  is  not  possible  to  have  fractional 
aircraft.  A  listing  of  all  of  the  types  of  aircraft  assigned  to  each  base  and  the  ranges  for 
each  factor  can  be  found  in  Table  5. 
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FACTOR 

MAX 

MIN 

TYPE 

BASE  A  -  RQ-4 

3 

0 

INTEGER 

BASE  A-RQ-1 

6 

0 

INTEGER 

BASE  A  -  P-3 

6 

0 

INTEGER 

BASE  A- RC- 135 

3 

0 

INTEGER 

BASE  A  -  U-2 

3 

0 

INTEGER 

BASE  B  -  RQ-4 

3 

0 

INTEGER 

BASE  B  -  RQ-1 

3 

0 

INTEGER 

BASE  B  -  P-3 

6 

0 

INTEGER 

BASE  B-RC- 135 

6 

0 

INTEGER 

BASE  B  -  U-2 

3 

0 

INTEGER 

BASE  C  -  RQ-4 

3 

0 

INTEGER 

BASE  C- RQ-1 

3 

0 

INTEGER 

BASE  C  -  P-3 

6 

0 

INTEGER 

BASE  C-RC- 135 

6 

0 

INTEGER 

BASE  C  -  U-2 

3 

0 

INTEGER 

BASE  D  -  RQ-4 

3 

0 

INTEGER 

BASED -RQ-1 

3 

0 

INTEGER 

BASE  D  -  P-3 

6 

0 

INTEGER 

BASE  D-RC- 135 

6 

0 

INTEGER 

BASE  D  -  U-2 

3 

0 

INTEGER 

Table  5.  Factors:  Assets  Assigned  to  Bases 

b.  Optimization  Interval 

Optimization  interval  is  the  elapsed  time  in  the  simulation  between 
optimization  events.  This  factor  can  take  on  any  continuous  value  greater  than  zero. 
Optimization  intervals  of  greater  than  24  hours  do  not  make  sense  operationally,  but  there 
is  nothing  that  prevents  an  analyst  from  choosing  a  longer  interval.  For  optimization 
intervals  that  exceed  the  set  scenario  length,  only  an  initial  optimization  would  be 
determined  and  the  simulation  would  run  to  completion  based  on  this  condition.  The 
range  of  the  optimization  interval  was  limited  to  between  0.5  and  6  hours  for  this 
experiment  (See  Table  6).  Preliminary  JDAFS  test  runs  indicate  that  optimization 
intervals  of  less  that  0.5  hours  result  in  excessive  runtimes. 


FACTOR 

MAX 

MIN 

TYPE 

OPTIMIZATION  INTERVAL 

6 

0.5 

CONTINUOUS 

Table  6.  Factor:  Optimization  Interval 


49 


2. 


NOLH 


Any  number  of  DOE  constructs  could  have  been  utilized  to  perfonn  this 
experiment,  such  as,  sequential  bifurcation,  folded  sequential  bifurcation,  X  p  fractional 
factorial,  combined  designs,  or  Nearly  Orthogonal  Latin  Hypercube  (NOLH)  designs, 
among  others  (Sanchez  2006;  Kleijnen  et  al.  2005).  However,  given  the  number  of 
factors  in  this  study,  not  all  of  the  DOE  applications  would  have  been  tractable.  Lor 
example,  a  full  factorial  design  with  only  3  levels  for  each  factor  would  result  in  321  or 
10,460,353,202  design  points.  Assuming,  only  the  deterministic  non-penetrating  scenario 
was  to  be  executed  and  each  run  took  a  nominal  5  minutes  to  complete,  the  total 
processing  time  required  would  be  nearly  1,000  years.  Lortunately,  alternatives  exist  that 
keep  the  run  time  at  a  practical  level  while  ensuring  a  meaningful  exploration  of  the 
response  surface. 

The  Nearly  Orthogonal  Latin  Hypercube  is  such  a  design.  The  NOLH  design 
provides  a  number  of  highly  desirable  DOE  qualities  including:  orthogonality,  space 
filling  and  bias  protection,  efficiency,  and  flexibility. 

•  Orthogonality10,  or  near  orthogonality,  is  important  to  ensure 
independence  of  the  regressors,  or  coefficient  estimates  in  regression 
models.  This  property  significantly  enhances  the  ability  to  evaluate  the 
suitability  of  factors  for  inclusion  in  the  resultant  metamodel  and 
determine  the  overall  contribution  to  metamodel  fit  (Kleijnen  et  al.  2005). 

•  Space-filling  designs  enable  the  exploration  of  all  regions  of  the  response 
surface,  not  just  the  edges.  By  distributing  the  design  points  throughout 
the  experimental  region,  analysts  can  minimize  the  number  of  assumptions 
made  about  the  response  surface.  Additionally,  “Space-filling  designs  also 
provide  flexibility  when  estimating  a  large  number  of  linear  and  nonlinear 
effects,  as  well  as  interaction,  and  so  provide  general  bias  protection  when 
fitting  metamodels  of  specific  forms”  (Kleijnen  et  al.  2005,  274). 

•  The  NOLH  design  provides  efficiency  in  allowing  the  rapid  detennination 
of  the  correct  minimum  number  of  design  points  necessary  for  an 
experiment.  Additionally,  with  the  use  of  a  NOLH  design  template,  an 
experimenter  can  quickly  produce  a  design  containing  numerous  factors 
with  multiple  or  continuous  levels. 


10 


Orthogonality  exists  when  the  inner  product  of  any  two  columns  of  a  matrix  is  equal  to  zero. 
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•  The  NOLH  design  is  extremely  flexible.  Factors  of  interest  can  rapidly  be 
added  or  levels  can  be  changed  to  provide  more  insight  into  a  model  or  to 
create  entirely  new  designs.  NOLH  designs  can  also  be  stacked  to  achieve 
more  desirable  pairwise  correlation  or  specific  design  points  of  interest 
can  be  appended  to  any  existing  design. 

For  this  study,  a  NOLH  DOE  was  constructed  using  the  NOLHDesigns_v4.xls 
worksheet  (See  Figure  22)  created  by  Dr.  Susan  Sanchez  (2005)  based  on  algorithms 
developed  by  Cioppa  (2002).  The  worksheets  have  been  crafted  specifically  to  avoid  the 
problems  associated  with  multicolinearity  and  ensure  a  robust  space-filling  design.  This 
worksheet,  capable  of  handling  up  to  29  factors,  is  available  at  <  http://harvest.nps.edu/>. 
Five  templates  are  included  in  the  workbook  to  facilitate  designs  with  various  numbers  of 
factors;  8-11,  12-16,  17-22,  and  23-29.  Each  template  can  be  used  for  any  number  of 
factors  up  to  the  maximum  allowed  in  the  template.  This  allows  smaller  experiments  to 
be  created  with  more  than  the  minimum  number  of  design  points,  improving  upon  the 
space-filling  properties  of  the  design  and  increasing  the  number  of  degrees  of  freedom  for 
the  output  data.  In  this  case,  the  23-29  factor  worksheet  was  used  to  create  257  design 
points11  based  on  the  21  factors  in  the  experiment. 


1 1  The  maximum  number  of  factors  in  a  Latin  Hypercube  is  determined  by  m  +  combination(m  - 1, 2) 
where  m  is  an  integer  greater  than  1.  Since  the  maximum  number  of  factors  for  the  chosen  design  template 
is  29,  solving  the  previous  equation  for  m,  yields  m  =  8.  The  number  of  design  points,  n,  can  be  found  by 
n  =  2"'  +1 .  Therefore,  a  design  that  includes  29  factors  requires  257  design  points. 
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E  Microsoft  Excel  -  DOE  WORKSHEET 


Figure  22.  Nearly  Orthogonal  Latin  Hypercube  Worksheet. 

The  design  created  consists  of  a  257  x  21  matrix  with  a  maximum  pairwise 
correlation  of  0.0942. 12’13  Figure  23  provides  a  scatterplot  of  the  design  factors 
demonstrating  the  space-filling  properties  of  the  NOLH  design.  The  uniform  appearance 
of  the  points  in  the  scatterplot  is  due  to  the  integer  structure  of  the  20  Assets  Assigned  to 


'  -  Ideally,  the  maximum  pairwise  correlation  should  be  less  than  0.03  to  classify  a  matrix  as  nearly 
orthogonal.  The  0.0942  achieved  in  the  design  for  this  study  is  low  enough  given  the  resolution  of  the 
model.  Creating  a  design  with  a  maximum  pairwise  correlation  with  the  factors  used  in  this  study  would 
have  required  more  than  doubling  the  number  of  design  points.  The  runtime  for  this  new  design  would 
have  been  impractical.  (Cioppa  2002). 

I3  Lower  correlation  numbers  can  be  achieved,  if  required,  by  creating  a  stacked  NOLH  design.  This 
is  done  by  pasting  the  values  from  the  NOLH  worksheet  into  a  new  worksheet.  Then,  randomly  permuting 
the  column  headings  in  the  original  NOLH  worksheet  and  pasting  those  values  into  the  new  worksheet 
containing  the  values  from  the  original  worksheet,  ensuring  that  the  values  in  the  columns  are  matched  to 
the  correct  column  headings.  This  results  in  a  design  with  the  same  number  of  columns,  but  twice  as  many 
rows  as  the  original  design. 
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Bases  factors.  The  space-filling  patterns  would  be  dramatically  different  if  the  factors 
were  continuous  in  nature.  An  additional  17  special  case  design  points  were  added, 
resulting  in  a  274  x  2 1  final  design  matrix. 

The  special  case  design  points  are  added  to  explore  the  response  at  specific 
combinations  of  factor  settings  that  were  not  captured  by  the  NOLH  design.  The  first  15 
design  points  represent  all  possible  combinations  of  1,  2,  3,  and  4  base  configurations 
with  maximum  aircraft  availability,  2 1  aircraft  at  each  open  base,  and  a  midpoint  setting 
of  the  optimization  interval,  3.25  hours.  The  2  remaining  design  points  are  for  4  base 
configurations  with  maximum  aircraft  availability  and  optimization  intervals  set  at  the 
minimum  and  maximum,  0.5  and  6  hours,  respectively. 
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Scatterplot  Matrix 


Figure  23 .  Scatterplot  Matrix  for  Design  Factors 

Additional  information  on  NOLH  can  be  found  in  Cioppa’s  (2002)  Efficient 
Nearly  Orthogonal  and  Space-Filling  Experimental  Design  for  Fligh-Dimensional 
Complex  Models  doctoral  dissertation. 
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C.  EXECUTION  OF  SCENARIOS 

1.  Implementing  DOE 

In  executing  an  experiment,  the  DOE  determines  the  number  of  design  points,  but 
the  number  of  replications  must  be  decided  upon  by  the  analyst.  A  deterministic  model 
needs  only  a  single  run  of  each  design  point.  No  matter  how  many  times  the  experiment 
is  run,  a  deterministic  model  always  returns  the  same  result.  Selecting  the  number  of 
replications  necessary  for  a  stochastic  model  is  more  challenging.  Unfortunately,  no 
clear  rules  exist  that  allow  a  detennination  of  the  “right”  number  of  replications  a  priori. 
The  number  of  replications  must  “gain  enough  data  to  achieve  narrow  confidence 
intervals  and  powerful  hypothesis  tests”  (Sanchez  2006,  69).  Conduct  too  few 
replications  and  the  experiment  may  have  to  be  rerun  to  get  more  data,  too  many 
replications  and  valuable  resources  may  have  been  wasted  (Chung  2004). 

For  the  purposes  of  this  study,  the  non-penetrating  scenario  is  purely  deterministic 
and  requires  only  one  replication  of  each  design  point.  The  penetrating  scenario, 
however,  has  the  stochastic  element  of  probability  of  kill  (Pk)  for  the  SAMs.  The  use  of 
10  replications  of  each  design  point  for  the  penetrating  scenario  was  all  that  was  feasible 
given  the  scope  of  this  research,  processing  constraints,  and  time  limitations. 

A  method  was  developed  for  JDAFS  so  that  the  analyst  can  specify  which  input 
factors  are  to  be  varied,  match  those  factors  with  a  design,  and  generate  the 
corresponding  input  database.  This  was  done  so  that  there  was  no  impact  whatsoever  on 
the  basic  input  database  design,  and  so  that  the  specific  experimental  design  used  was 
completely  interchangeable.  That  is,  the  analyst  could  keep  the  same  basic  input  scenario 
and  same  set  of  factors,  but  swap  in  any  experimental  design  scheme  that  was  desired 
with  only  a  minimal  amount  of  changes.  For  each  design  point,  the  DesignGenerator 
(See  Figure  24)  program  uses  SQL  statements  to  create  a  corresponding  input  file. 
JDAFS  writes  its  output  reports  to  the  same  database  that  was  used  for  input,  so  there  is 
always  a  direct  association  between  input  and  output  values. 
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Execution  of  JDAFS  is  oriented  towards  a  single  run  with  a  single  input  file;  there 
is  no  built-in  capability  in  JDAFS  itself  to  run  experimental  designs.  Yet  executing 
JDAFS  with  various  input  factors  set  based  on  efficient  designs  is  essential  to  its  effective 
use.  The  normal  input  to  a  JDAFS  scenario  is  currently  via  an  Access  database;  support 
for  other  databases  such  as  MySQF,  Oracle,  and  Derby  will  be  added  to  future  versions 
of  the  program. 

Setting  a  factor  requires  more  than  changing  a  single  value;  all  data  elements  in 
columns  corresponding  to  that  factor  must  be  changed  to  correctly  set  the  factor  to  its 
appropriate  level.  In  order  to  use  the  DesignGenerator,  the  values  in  the  NOFH 
worksheet  must  be  nonnalized  to  a  range  of  [0,  1]  and  saved  in  plain-text  comma- 
separated  (csv)  files.  Conceptually,  these  files  constitute  tables  in  a  virtual  database  (and 
future  versions  will  be  implemented  as  such);  therefore,  these  tables  of  values  are 
represented  as  a  database  called  “Designs.”  Different  schemes  can  be  used  to  generate 
these  tables  of  values  without  modifying  any  other  part  of  the  input  or  the  code. 

The  Factors  database  consists  of  two  tables:  one  to  identify  factors  in  the  template 
database  by  table  and  column,  together  with  a  user-specified  minimum  and  maximum 
value;  and  a  second  table  that  maps  the  number  of  factors  (determined  by  the  number  of 
rows  in  the  first  table)  to  one  of  the  design  point  files.  That  table  is  then  read,  and  the 
corresponding  values  applied  to  the  minimum  and  maximum  values  to  produce  an  input 
value.  The  “DesignGenerator”  program  that  does  this  is  written  in  Java  and  uses  Java 
Database  Connectivity  (JDBC)  to  access  the  databases.  Hence,  using  different  databases 
only  requires  that  a  different  JDBC  driver  be  installed. 

For  each  design  point,  the  DesignGenerator  program  uses  SQF  statements  to 
create  a  corresponding  input  file,  which  is  numbered  <name>xxxx.mdb,  where  “xxxx”  is 
an  index  corresponding  to  the  particular  design  point  index.  All  the  input  from  the 
template  database  is  copied,  and  those  entries  that  are  factors  are  modified  using  the  SQF 
UPDATE  query  to  be  set  to  the  design  point  values.  An  additional  table  is  written  that 
identifies  the  particular  names  and  values  of  the  input  factors.  This  information  is  used 
after  JDAFS  is  executed  to  extract  the  independent  and  dependent  variables  for  statistical 
analysis. 
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Figure  24.  Design  Generator  for  Producing  Input  Files 


2.  Computing  Resources 

While  JDAFS  is  capable  of  running  on  standalone  laptop  or  desktop  computers,  it 
is  impractical  to  run  simulation  experiments  requiring  hundreds  or  possibly  millions  of 
runs,  each  potentially  requiring  several  hours  of  runtime,  on  anything  but  a  computing 
cluster.  For  this  particular  experiment,  the  non-penetrating  scenario  required  274  runs 
and  the  penetrating  scenario  required  2740  runs  (274  x  10).  These  production  runs  were 
executed  by  Dr.  Paul  Sanchez,  Senior  Lecturer  in  the  Operations  Research  Department  at 
the  Naval  Postgraduate  School,  on  the  Simulation  Experiments  &  Efficient  Designs 
(SEED)  Center  Microsoft  Windows  computing  cluster  located  at  TRAC-Monterey  on  the 
NPS  campus  in  Monterey,  CA.  The  SEED  Center  computing  cluster  is  composed  of  12 
dual  processor  Dell  desktops  with  2G  of  RAM,  running  Windows  XP.  The  machines  are 
using  Condor  cluster  software,  which  is  open  source  software  available  from  the 
University  of  Wisconsin  <http://www.cs.wisc.edu/condor/>.  The  individual  computers 
have  Cygwin  software  <http://  www.cygwin.com/>  installed  to  enable  improved 
command  line  tools,  remote  access  via  Secure  Shell,  and  sophisticated  scripting.  The 
non-penetrating  scenario  runs  were  completed  in  approximately  4  hours  and  the 

penetrating  scenario  runs  in  approximately  60  hours. 
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3.  Post-processing  of  JDAFS  Output 

Upon  completion  of  the  JDAFS  model  runs  on  the  SEED  Center  computing 
cluster,  the  output  data  is  written  to  tables  in  the  input  database.  Therefore,  274  separate 
output  databases  are  created,  one  for  each  design  point  in  the  scenario.  By  writing  the 
output  reports  to  the  same  database  that  was  used  for  input,  there  is  always  a  direct 
association  between  input  and  output  values.  The  independent  input/output  databases 
must  be  combined  into  a  consolidated  database  prior  to  performing  any  further  analysis. 

The  process  for  combining  the  independent  output  databases  into  a  single 
common  output  database  is  essentially  the  reverse  of  creating  the  design  databases.  An 
OutputGenerator  (See  Figure  25)  was  created  in  JAVA  using  SQL  statements  to  create  a 
single  output  database.  Like  the  DesignGenerator,  JDBC  is  used  to  access  the  databases 
in  the  OutputGenerator.  The  OutputGenerator  appends  the  information  in  the  individual 
output  database  tables  to  like  tables  in  the  common  output  database.  Since  the  JDAFS 
output  data  is  written  in  string  format,  individual  columns  of  data  that  should  be  numeric 
must  be  converted  to  the  proper  format  prior  to  any  data  analysis. 


OutputOOOO.mdb 


Output0274.mdb 


Output 

Generator 


Output.mdb 


Figure  25 .  Output  Generator  for  Producing  Output  Files 
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The  cumulative  size  of  the  output  databases  at  the  completion  of  the  simulation 
runs  was  over  7.5  gigabytes.  The  process  of  combining  the  output  databases  into  a  single 
database  for  each  scenario  reduced  the  overall  file  size  to  approximately  470  megabytes. 
This  was  further  reduced  to  just  over  300  megabytes  by  converting  the  output  database 
tables  into  tables  in  JMP. 
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V.  ANALYSIS 


A  detailed  pre-simulation  analysis  of  the  scenarios  and  a  post-simulation  analysis 
of  the  output  data  collected  from  the  execution  of  this  study’s  penetrating  and  non¬ 
penetrating  scenarios  were  conducted.  The  following  chapter  describes  the  analysis 
process  and  relevant  discoveries  and  points  of  interest  from  this  research. 

A.  ANALYSIS  METHODOLOGY,  TECHNIQUES,  AND  TOOLS 

1.  Methodology 

The  same  analysis  methodology  is  applied  to  both  scenarios,  non-penetrating  and 
penetrating,  in  this  experiment.  Pre- simulation  spreadsheet  analysis  was  conducted  to 
gain  an  understanding  of  how  the  scenario  design  would  impact  the  JDAFS  simulation 
results  and  to  provide  the  capability  to  ensure  reasonableness  of  JDAFS  output.  Upon 
completion  of  the  simulations,  the  data  from  each  scenario  was  used  to  generate  summary 
statistics  to  verify  the  validity  of  the  output  and  identify  outlying  data  points.  The 
outliers  were  examined  to  determine  their  cause  and  potential  impact  on  subsequent 
analysis.  Multiple  regression  analysis  was  performed  to  allow  the  construction  of 
metamodels  for  each  scenario.  The  validity  of  these  metamodels  was  confirmed  through 
the  use  of  regression  and  classification  trees.  Additional  graphical  analysis  techniques 
were  perfonned  to  gain  insights  and  understanding  into  the  behavior  and  influence  of 
specific  factors. 

2.  Analytical  Techniques 

A  variety  of  analytical  techniques  were  applied  to  fully  examine  and  understand 
the  scenarios  and  output  from  this  study.  The  various  techniques  are  complementary  to 
one  another  and  serve  to  validate  the  results  of  the  methodology.  The  techniques  applied 
in  this  study  were  spreadsheet,  graphical,  multiple  regression,  and  classification  and 
regression  tree  (CART)  analysis. 
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a.  Spreadsheet  Analysis 

Spreadsheet  analysis  enables  the  rapid  examination,  manipulation,  and 
display  of  large  amounts  of  data  in  a  flexible  environment.  The  use  of  this  tool  facilitated 
the  construction  and  evaluation  of  a  variety  of  easily  recon  figurable  scenarios  that  were 
compatible  with  the  JDAFS  input  databases  and  more  sophisticated  analysis  tools.  While 
not  suitable  for  complex  post-simulation  data  analysis,  the  use  of  spreadsheets  enables  a 
means  to  examine  design  scenarios  prior  to  conducting  simulation  runs. 

b.  Multiple  Regression 

Regression  is  a  technique  that  constructs  a  probabilistic  model  to  examine 
the  relationship  between  a  dependent  (response)  variable  and  an  independent  variable 
(predictor)  (Montgomery  et  al.  2004).  Multiple  regression  involves  more  than  one 
regressor  or  predictor  and  may  contain  interaction  and  quadratic  terms.  Models  with  first 
order  predictors,  interaction,  and  quadratic  terms  are  referred  to  as  full  quadratic  models. 
The  response  variable  is  represented  as  Y  and  the  predictor  variables  as  xl,x2,...,xk , 
where  k  indicates  the  number  of  regressors.  The  j5  parameters,  are 

regression  coefficients  that  detennine  the  mean  amount  of  change  in  Y  for  every  one  unit 
change  in  the  associated  predictor  variable  while  all  other  predictors  are  held  constant. 
The  j30  parameter  is  a  constant  term  that  indicates  the  point  at  which  the  regression  line 

intersects  the  y-axis.  The  error  term,  s ,  is  a  “random  variable  that  accounts  for  the 
failure  of  the  model  to  fit  the  data  exactly”  (Montgomery  et  al.  2004,  1).  Also  known  as 
the  residual,  sf  is  assumed  to  be  normally  distributed  with  a  mean  of  0  and  a  variance  of 

a2  or  A(0,cr2).  Devore  (2004,  588)  illustrates  four  common  forms  for  multiple 
regression  models  containing  two  independent  variables,  v,  and  x2 . 

1 .  First  order  model: 

Y  =  J30  +  /?jXj  +  y 3^x i  +  s 

2.  Second  order  no-interaction  model: 

Y  =  th  +  P\X\  +  PlX2  +  P\X\  +  PaX2  +  S 
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3. 


Model  with  first  order  predictors  and  interaction: 


Y  =  Po  +  M  +  Pix  2  +  #1*1*2  +  s 

4.  Full  quadratic  model: 

Y  =  fio  +  P\X\  +  PlX  2  +  P\X\  +  #4*2  +  #5*1*2  +  £ 

Regression  models  must  conform  to  several  assumptions  in  order  to  be 
considered  valid  (Montgomery  et  al.  2004,  131): 

1.  The  relationship  between  the  response  y  and  the  regressors  is  linear,  at 
least  approximately. 

2.  The  error  term  s  has  a  zero  mean. 

3.  The  error  term  s  has  a  constant  variance  a2 . 

4.  The  errors  are  uncorrelated. 

5.  The  errors  are  normally  distributed. 

These  assumptions  are  summarized  as  follows: 

iid  . 

si  ~ iV(0,  a2),  Vi. 

Conformity  to  these  assumptions  is  demonstrated  for  each  of  the 
regression  models  constructed  in  this  study.  The  linearity  assumption  between  the 
response  variable  y  and  the  regressors  is  checked  by  plotting  the  regression  line  against  a 
scatter  plot  of  the  data,  an  actual  by  predicted  plot.  The  assumptions  of  a  zero  mean, 
constant  variance  of  a2 ,  and  uncorrelated  errors  are  confirmed  through  a  residual  by 
predicted  plot.  This  is  indicated  by  a  mean  of  zero  and  a  cloud  of  apparently  random  data 
points.  Finally,  the  normality  assumption  is  demonstrated  by  histogram  plot  of  the 
residuals  overlaid  with  a  normal  curve. 
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For  the  purposes  of  this  research,  up  to  full  quadratic  models  are 
constructed  by  stepwise  regression.  The  wide  variety  of  distances  in  mission  to  mission 
area  pairings  along  with  the  differences  in  aircraft  and  sensor  operational  parameters 
result  in  inherent  nonlinearities.  These  complexities  must  be  accounted  for  through  the 
use  of  full  quadratic  models. 

c.  Classification  and  Regression  Trees  (CART) 

Classification  and  regression  trees  provide  a  method  for  recursively 
partitioning  data  sets  in  accordance  with  the  relationship  between  the  response  variable 
and  the  predictors.  Each  split  creates  two  branches  in  an  inverted  tree  structure  by 
considering  all  possible  cuts  or  groupings  and  selecting  the  partition  with  the  largest 
likelihood-ratio  chi-square  statistic  (JMP  Statistical  Discovery  Software  2006).  This 
method  generates  a  chart  that  is  straightforward  to  interpret  and  clearly  shows  the  most 
significant  factors. 

d.  Graphical  Analysis 

Graphical  analysis  provides  a  visual  representation  of  data  allowing  for 
rapid  interpretation  and  enhanced  understanding.  Overlays,  bar  charts,  scatter  plots,  line 
graphs,  and  contour  plots  were  all  use  in  the  exploration  of  the  pre-  and  post-simulation 
data.  Additionally,  the  graphical  representation  of  relevant  findings  is  of  benefit  in 
conveying  complex  infonnation  to  individuals  or  groups  not  intimately  familiar  with  this 
topic  or  the  research  methods  applied. 

3.  Analysis  Tools 

a.  Microsoft  Excel 

Microsoft  Excel  was  used  to  perform  pre-simulation  spreadsheet  analysis 
of  the  scenarios  developed  for  this  experiment  in  order  to  understand  the  type  of  output 
that  might  be  generated  by  the  actual  JDAFS  production  runs.  Additionally,  Excel  was 
utilized,  where  appropriate,  for  the  production  of  explanatory  tables  and  graphs  in  the 
analysis  of  the  simulation  output. 
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b.  JMP  Statistical  Discovery  Software 

The  JMP  (pronounce  ‘jump’)  Statistical  Discovery  Software  from  SAS  is 
the  tool  chosen  for  analysis  of  the  resultant  data  from  the  JDAFS  model  runs.  This 
powerful  analysis  tool  is  intuitive  to  use  and  does  not  require  additional  programming  or 
script  writing  to  obtain  results.  JMP  allows  the  analyst  to  interactively  investigate  data  in 
a  dynamically  linked  spreadsheet  and  graphical  environment.  Additionally,  JMP 
provides  a  journaling  function  for  saving  and  revisiting  graphics  or  data  tables  of  interest. 
More  information  on  JMP  Statistical  Discovery  Software  can  be  found  on  the  company’s 
web  site:  <http://www.jmp.com>. 

B.  PRE-SIMULATION  ANALYSIS 
1.  Non-Penetrating  Scenario 

The  non-penetrating  scenario  limits  the  collection  ability  of  the  ISR  platforms  to 
standoff  mission  areas.  This  restriction  causes  the  implementation  of  waypoint  routing  to 
prevent  COI  overflight,  thereby  requiring  additional  flight  time  between  bases  and 
mission  areas. 


a.  Scenario  Geometry 

As  shown  in  Chapter  III,  Table  1,  there  are  25  targets,  each  with  up  to  3 
sensor  requirements.  These  25  targets  are  broken  down  into  20  EO/IR,  18  SAR,  and  17 
SIGINT  requirements,  for  a  total  of  55  missions.  These  55  missions  or  sensor 
requirements  can  be  serviced  from  any  of  the  14  mission  areas  in  the  non-penetrating 
scenario.  ( Satisfaction  of  sensor  requirements  depends  upon  the  availability  an  ISR 
platform  with  the  correct  sensor  package,  sufficient  sensor  range  and  the  platform ’s 
ability  to  meet  the  minimum  altitude  to  achieve  line-of-sight.)  By  matching  the  14 
mission  areas  with  the  55  missions,  770  possible  mission  area-to-mission  pairings  are 
possible;  280  EO/IR,  252  SAR,  and  238  SIGINT.  See  Figure  26  for  an  example  of 
mission  area-to-mission  matching.  Each  of  these  pairings  is  taken  into  account  by 
JDAFS  in  detennining  the  optimum  allocation  of  ISR  platforms. 
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Figure  26.  Example  of  Mission  Area-to-Mission  Pairings  for  a  Single  Mission  Area 
to  all  EO/IR  Missions 


It  is  expected  that  the  lowest  absolute  distance  and  the  lowest  mean  and 
median  distance  from  a  base  to  its  associated  mission  areas  will  play  a  significant  role  in 
determining  the  overall  coverage  achieved  by  the  various  ISR  platforms.  An  examination 
of  the  basing  configuration  as  shown  in  Figure  27  and  Table  7  reveals  the  following: 

•  Base  D  is  closest  to  its  nearest  mission  area,  then  A,  C,  and  B, 
respectively. 

•  Base  C  has  the  lowest  maximum  distance  to  its  farthest  mission  area,  then 
D,  A,  and  B,  respectively. 

•  Base  D  has  the  lowest  mean  and  median  travel  distance  to  its  mission 
areas,  then  C,  A,  and  B,  respectively. 

Note  that  waypoint  routing  affects  the  mean  and  median  travel  distance  in  the  non¬ 
penetrating  scenario  since  the  COI  airspace  cannot  be  violated. 


Figure  27.  Basing  Configuration 
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SUMMARY  OF  BASE  TO  MISSION 

AREA  DISTANCE  STATISTICS 

A 

B 

C 

D 

MIN 

328000 

936862 

578000 

187840 

MAX 

1854124 

1899964 

1390502 

1760154 

MEAN 

974987 

1271025 

938075 

875094 

MEDIAN 

932023 

1197693 

920821 

842965 

All  distances  are  in  meters 

Table  7.  Non-Penetrating  Scenario  Summary  of  Base  to  Mission  Area  Distance 
Statistics  (The  additional  transit  required  due  to  waypoint  routing  to  prevent 
COI  overflight  is  included  in  the  distance  calculations  for  this  table). 

b.  Sensor  Coverage 

Each  of  the  ISR  platforms  has  a  differing  ability  meet  the  various  sensors 
demands.  The  requirement  to  collect  from  standoff  mission  areas  in  the  non-penetrating 
scenario  limits  the  amount  of  coverage  each  aircraft  can  provide.  The  following  figures 
depict  the  potential  coverage  by  each  type  of  platform  and  sensor14. 

Figure  28  illustrates  the  potential  coverage  by  the  RQ-1  EO/IR  and  SAR 
sensors.  There  are  no  EO/IR  targets  and  ten  SAR  targets  that  can  be  ranged  by  the  RQ-1 . 


Figure  28.  RQ-1  Sensor  Coverage  for  the  Non-Penetrating  Scenario 


14  Note  that  the  amount  of  coverage  depicted  in  the  illustrations  may  differ  slightly  from  the  amount  of 
coverage  that  has  been  calculated  due  to  the  imprecise  nature  of  the  graphics.  In  all  cases,  the  text  refers  to 
the  calculated  coverage. 
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Figure  29  demonstrates  the  ability  of  the  RQ-4  to  cover  EO/IR,  SAR  and 
SIGINT  sensor  requirements.  The  RQ-4  can  satisfy  2  EO/IR,  10  SAR,  and  13  SIGINT 
requirements. 


Figure  29.  RQ-4  Sensor  Coverage  for  the  Non-Penetrating  Scenario 

The  ability  of  the  P-3  to  provide  EO/IR  and  SAR  coverage  is  illustrated  in 
Figure  30.  The  P-3  can  range  2  EO/IR  missions  and  10  SAR  missions. 


Figure  30.  P-3  Sensor  Coverage  for  the  Non-Penetrating  Scenario 

Figure  31  depicts  the  coverage  of  the  RC-135.  The  RC-135  has  the  ability 
to  cover  13  SIGINT  sensor  requirements. 
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Figure  3 1 .  RC-135  Sensor  Coverage  for  the  Non-Penetrating  Scenario 

The  U-2’s  sensor  capabilities  are  shown  in  Figure  32.  The  U-2  can  satisfy 
7  EO/IR,  11  SAR,  and  13  SIGINT  missions,  making  it,  sensor  wise,  the  most  capable 
platform  in  the  scenario. 


Figure  32.  U-2  Sensor  Coverage  for  the  Non-Penetrating  Scenario 

Table  8  provides  a  summary  of  the  number  of  missions  that  can  be 
covered  by  each  ISR  platform  from  each  mission  area.  Ranked  by  ability  to  provide 
coverage  of  the  most  targets  per  mission  area,  the  most  capable  aircraft  is  the  U-2, 
followed  by  the  RQ-4,  the  RC-135,  the  P-3,  and  finally  the  RQ-1.  This  further 
demonstrates  the  U-2’s  superior  ability  to  provide  coverage.  Note  that  the  capability  of 
the  RQ-1  and  the  P-3  is  very  similar,  with  the  P-3  marginally  exceeding  the  RQ-1. 
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MISSION 

AREA 

RQ-4 

RQ-1 

P-3 

RC-135 

U-2 

EO/IR 

SAR 

SIGINT 

TOTAL 

EO/IR 

SAR 

TOTAL 

EO/IR 

SAR 

TOTAL 

SIGINT/TOTAL 

EO/IR 

SAR 

SIGINT 

TOTAL 

i 

t 

t 

5 

7 
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1 
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1 

1 

2 
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1 

2 

6 
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2 

0 

2 

3 

5 

0 

2 

2 

0 

2 

2 

3 

1 

2 

6 

9 

3 

0 

1 

3 

4 

0 

1 

1 

0 

1 

1 

3 

1 

1 

5 

7 

4 

0 

1 

4 

5 

0 

1 

1 

0 

1 

1 

4 

0 

2 

7 

9 

5 

0 

2 

5 

7 

0 

2 

2 

0 

2 

2 

5 

0 

3 

8 

11 

6 

1 

2 

4 

7 

0 
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3 

4 

1 

3 

6 

10 

7 

0 

3 

2 

5 

0 

3 

3 

0 

3 

3 

2 

1 

3 

6 

10 

8 

0 

3 

4 

7 

0 

3 

3 

0 

3 

3 

4 

1 

4 

6 

11 

9 

0 

3 

2 

5 

0 

3 

3 

0 

3 

3 

2 

1 

4 

5 

10 

10 

0 

2 

1 

3 

0 

2 

2 

0 

2 

2 

1 

1 

2 

4 

7 

11 

0 

1 

1 

2 

0 

1 

1 
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1 

1 

1 

0 

1 

2 

3 

12 

0 

1 

1 

2 

0 

1 
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1 

1 

0 

3 

2 

5 

13 

0 

2 

1 

3 

0 

2 

2 

0 

2 

2 

1 

1 

2 

1 

4 

24 

0 

1 

3 

4 

0 

1 

1 

0 

1 

1 

3 

0 

1 

6 

7 

TOTAL 

2 

25 

39 

66 

0 

25 

25 

2 

25 

27 

39 

9 

33 

70 

112 

MIN 

0 

1 

1 

2 

0 

1 

1 

0 

1 

1 

1 

0 

1 

1 

3 

MAX 

1 

3 

5 

7 

0 

3 

3 

1 

3 

3 

5 

1 

4 

8 

11 

MEAN 

0.1 

1.8 

2.8 

4.7 

0.0 

1.8 

1.8 

0.1 

1.8 

1.9 

2.8 

0.6 

2.4 

5.0 

8.0 

MEDIAN 

0 

2 

3 

5 

0 

2 

2 

0 

2 

2 

3 

1 

2 

6 

9 

Table  8.  Non-Penetrating  Scenario:  Number  of  Missions  Potentially  Satisfied  from 
Each  Mission  Area  by  Platform  and  Sensor  Type 

Figure  33  compares  the  ability  of  the  various  ISR  platforms  to  provide 
coverage.  Note  that  the  U-2  is  superior  to  all  other  aircraft  in  every  category.  This 
difference  is  due  to  the  U-2’s  advantage  in  sensor  range  for  all  types  of  sensors. 


Figure  33 .  Potential  Mission  Coverage  by  ISR  Platform 
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Figure  34  illustrates  the  predicted  time  on  station  for  each  ISR  platform 
from  each  base.  The  RQ-4  with  its  comparatively  high  speed  and  36  hour  mission 
duration  is  the  clear  leader  in  the  ability  to  remain  on-station.  Interestingly,  the  RQ-1 
provides  the  least  amount  of  time  on-station  per  sortie.  Despite  a  24  hour  mission 
duration,  the  RQ-4’s  on-station  endurance  is  limited  by  its  slow  speed.  It  is  probable  that 
the  ability  to  cover  a  greater  number  of  targets  per  sortie  will  be  more  important  to 
overall  coverage  than  the  ability  to  remain  on-station  for  a  longer  period  of  time. 


The  ability  to  satisfy  all  of  the  missions  in  the  scenario  is  shown  in  Table 
9.  There  are  13  of  20  EO/IR,  7  of  18  SAR,  and  4  of  17  SIGINT  requirements  that  cannot 
be  met  by  any  platform.  This  indicates  that  only  about  44%  of  the  sensor  requirements  in 
the  non-penetrating  scenario  can  be  satisfied.  This  does  not  mean  that  44%  is  the 
maximum  coverage  number  that  can  be  achieved  in  the  simulation.  Since  ISR  platforms 
receive  credit  for  all  missions  that  are  ranged  regardless  of  whether  or  not  they  are 
assigned  to  that  mission,  the  total  coverage  number  will  likely  be  higher  than  44%  due  to 
duplication  of  mission  coverage. 
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EO/IR  Sensor  Requirements 

Mission 

RQ-4 

RQ-1 

P-3 

RC-135  U-2  Any 

1.1 

***  NO  SENSOR  REQUIREMENT 

*** 

2.1 

0 

0 

0 

0 

■ 

3.1 

0 

0 

0 

0 

°l 

4.1 

1 

0 

1 

0 

1 

3 

5.1 

0 

0 

0 

0 

■ 

6.1 

0 

0 

0 

0 

2 

2 

7.1 

***  NO  SENSOR  REQUIREMENT 

*** 

8.1 

***  NO  SENSOR  REQUIREMENT 

*** 

9.1 

0 

0 

0 

0 

1 

1 

10.1 

0 

0 

0 

0 

°l 

11.1 

***  NO  SENSOR  REQUIREMENT 

*** 

12.1 

0 

0 

0 

0 

■ 

13.1 

0 

0 

0 

0 

°l 

14.1 

1 

0 

1 

0 

1 

3 

15.1 

0 

0 

0 

0 

°l 

16.1 

0 

0 

0 

0 

2 

2 

17.1 

0 

0 

0 

0 

1 

1 

18.1 

0 

0 

0 

0 

°l 

19.1 

0 

0 

0 

0 

ol 

20.1 

0 

0 

0 

0 

°l 

21.1 

0 

0 

0 

0 

1 

1 

22.1 

0 

0 

0 

0 

°l 

23.1 

***  NO  SENSOR  REQUIREMENT 

*** 

24.1 

0 

0 

0 

0 

■ 

25.1 

0 

0 

0 

0 

ol 

SAR  Sensor  Requirements  j 

Mission 

RQ-4 

RQ-1 

P-3 

RC-135 

U-2  Any  | 

1.1 

***  NO  SENSOR  REQUIREMENT 

*** 

2.1 

0 

0 

0 

0 

oH 

3.1 

***  NO  SENSOR  REQUIREMENT 

*** 

4.1 

3 

3 

3 

0 

3 

12| 

5.1 

0 

0 

0 

0 

°l 

6.1 

3 

3 

3 

0 

4 

13| 

7.1 

***  NO  SENSOR  REQUIREMENT 

*** 

8.1 

***  NO  SENSOR  REQUIREMENT 

*** 

9.1 

3 

3 

3 

0 

3 

12 

10.1 

1 

1 

1 

0 

2 

5 

11.1 

***  NO  SENSOR  REQUIREMENT 

*** 

12.1 

0 

0 

0 

0 

2 

2] 

13.1 

0 

0 

0 

0 

°l 

14.1 

3 

3 

3 

0 

4 

13| 

15.1 

0 

0 

0 

0 

°l 

16.1 

3 

3 

3 

0 

4 

13] 

17.1 

3 

3 

3 

0 

4 

13 

18.1 

***  NO  SENSOR  REQUIREMENT 

*** 

19.1 

3 

3 

3 

0 

3 

12 

20.1 

2 

2 

2 

0 

2 

8 

21.1 

1 

1 

1 

0 

2 

5 

22.1 

0 

0 

0 

0 

■ 

23.1 

***  NO  SENSOR  REQUIREMENT 

*** 

24.1 

0 

0 

0 

0 

oM 

25.1 

0 

0 

0 

0 

o| 

SIGINT  Sensor  Requirements 


Mission 

RQ-4 

RQ-1 

P-3 

RC-135 

U-2 

Any 

1.3 

2 

0 

0 

2 

3 

7 

2.3 

1 

0 

0 

1 

2 

4 

3.3 

***  NO  SENSOR  REQUIREMENT  *** 

4.3 

4 

0 

0 

4 

5 

13 

5.3 

1 

0 

0 

1 

6 

8 

6.3 

5 

0 

0 

5 

6 

16 

7.3 

5 

0 

0 

5 

7 

17 

8.3 

0 

0 

0 

0 

0 

9.3 

4 

0 

0 

4 

6 

14 

10.3 

3 

0 

0 

3 

7 

13 

11.3 

2 

0 

0 

2 

6 

10 

12.3 

***  NO  SENSOR  REQUIREMENT  *** 

13.3 

***  NO  SENSOR  REQUIREMENT  *** 

14.3 

5 

0 

0 

5 

6 

16 

15.3 

1 

0 

0 

1 

6 

8 

16.3 

4 

0 

0 

4 

6 

14 

17.3 

***  NO  SENSOR  REQUIREMENT  *** 

18.3 

***  NO  SENSOR  REQUIREMENT  *** 

19.3 

***  NO  SENSOR  REQUIREMENT  *** 

20.3 

***  NO  SENSOR  REQUIREMENT  *** 

21.3 

2 

0 

0 

2 

4 

8 

22.3 

***  NO  SENSOR  REQUIREMENT  *** 

23.3 

0 

0 

0 

0 

0 

0 

24.3 

0 

0 

0 

0 

0 

0 

25.3 

0 

0 

0 

0 

0 

0 

Table  9.  Non-Penetrating  Scenario:  Ability  to  Satisfy  Sensor  Requirements  by 
Platform  Type  (Cells  highlighted  in  red  indicate  that  the  sensor  requirement 
cannot  be  met  by  any  platfonn  type) 


c.  Assessment 


The  following  assessments  are  made  based  on  the  preceding  analysis  of 
the  factors  affecting  the  non-penetrating  scenario: 

•  The  U-2’s  ability  to  cover  the  most  missions  per  sortie  coupled  with  its 
relatively  high  speed  will  make  it  the  most  significant  asset  in  the 
simulation. 

•  The  RQ-4  will  be  important  in  maximizing  coverage  due  to  its  long 
mission  duration  and  relatively  strong  sensor  package. 

•  The  relatively  long  range  of  the  SIGINT  sensor  and  the  second  best  on- 
station  perfonnance  may  make  the  RC-135  a  significant  contributor  to 
overall  coverage. 

•  The  P-3’s  relatively  short  sensor  ranges  and  the  lack  of  a  SIGINT  sensor 
will  limit  its  contribution  to  mission  coverage. 

•  It  is  expected  that  the  inability  to  cover  any  EO/IR  targets  will  limit  the 
utilization  of  the  RQ- 1  within  the  simulation. 

•  The  collective  inability  to  range  over  50%  of  the  missions  will  severely 
limit  the  mean  coverage. 
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2.  Penetrating  Scenario 

The  penetrating  scenario  allows  the  violation  of  the  COI’s  airspace,  removing  the 
standoff  limitation  and  adding  12  additional  mission  areas.  Waypoint  routing  is  no 
longer  uesed;  however,  the  ISR  platforms  are  now  vulnerable  to  air  defense  threats. 

a.  Scenario  Geometry 

As  shown  in  Chapter  III,  Table  1,  there  are  25  targets,  each  with  up  to  3 
sensor  requirements.  These  25  targets  are  broken  down  into  20  EO/IR,  18  SAR,  and  17 
SIGINT  requirements,  for  a  total  of  55  missions.  These  55  missions  or  sensor 
requirements  can  be  serviced  from  any  of  the  26  mission  areas  in  the  penetrating 
scenario.  ( Satisfaction  of  sensor  requirements  depends  upon  the  availability  an  ISR 
platform  with  the  correct  sensor  package,  sufficient  sensor  range  and  the  platform ’s 
ability  to  meet  the  minimum  altitude  to  achieve  line-of-sight.)  By  matching  the  26 
mission  areas  with  the  55  missions,  1430  possible  mission  area-to-mission  pairings  are 
possible;  520  EO/IR,  468  SAR,  and  442  SIGINT.  See  Figure  26,  page  65  for  an  example 
of  mission  area-to-mission  matching.  Each  of  these  pairings  is  taken  into  account  by 
JDAFS  in  detennining  the  optimum  allocation  of  ISR  platforms. 

Just  as  in  the  non-penetrating  scenario,  it  is  expected  that  the  lowest 
absolute  distance  and  the  lowest  mean  and  median  distance  from  a  base  to  its  associated 
mission  areas  will  play  a  significant  role  in  determining  the  overall  coverage  achieved  by 
the  various  ISR  platforms.  An  examination  of  the  basing  configuration  as  shown  in 
Figure  35  and  Table  10  reveals  the  following: 

•  Base  D  is  closest  to  its  nearest  mission  area,  then  A,  C,  and  B, 
respectively. 

•  Base  D  has  the  lowest  maximum  distance  to  its  farthest  mission  area,  then 
A,  C,  and  B,  respectively. 

•  Base  D  has  the  lowest  mean  and  median  travel  distance  to  its  mission 
areas,  then  A,  C,  and  B,  respectively. 

Since  no  waypoint  routing  is  necessary,  the  relative  distances  are  constant  from  each  base 
to  its  mission  areas.  That  is,  not  only  is  Base  D  the  closest  to  its  nearest  mission  area,  it 
also  has  the  lowest  mean  distance  to  its  mission  areas. 
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□  11 


Figure  35.  Basing  Configuration 


SUMMARY  OF  BASE  TO  MISSION 

AREA  DISTANCE  STATISTICS 

A 

B 

C 

D 

MIN 

328000 

936862 

578000 

187840 

MAX 

1398315 

1790219 

1500000 

1268418 

MEAN 

877152 

1345442 

1018558 

764431 

MEDIAN 

901274 

1336872 

1036622 

776566 

All  distances  are  in  meters 

Table  10.  Penetrating  Scenario  Summary  of  Base  to  Mission  Area  Distance  Statistics 
b.  Sensor  Coverage 

Each  of  the  ISR  platforms  has  a  differing  ability  meet  the  various  sensors 
demands.  The  requirement  to  collect  from  standoff  mission  areas  in  the  non-penetrating 
scenario  limits  the  amount  of  coverage  each  aircraft  can  provide.  The  following  figures 
depict  the  potential  coverage  by  each  type  of  platform  and  sensor.15 

Figure  36  illustrates  the  potential  coverage  by  the  RQ-1  EO/IR  and  SAR 
sensors.  There  are  3  EO/IR  targets  and  18  SAR  targets  that  can  be  ranged  by  the  RQ-1. 
Note  that  the  RQ-1  can  provide  SAR  coverage  of  the  entire  COI  with  the  exception  of  a 
small  area  in  the  southwest  corner. 


15  Note  that  the  amount  of  coverage  depicted  in  the  illustrations  may  differ  slightly  from  the  amount  of 
coverage  that  has  been  calculated  due  to  the  imprecise  nature  of  the  graphics.  In  all  cases,  the  text  refers  to 
the  calculated  coverage. 
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Figure  36.  RQ-1  Sensor  Coverage  for  the  Penetrating  Scenario 

Figure  37  demonstrates  the  ability  of  the  RQ-4  to  cover  EO/IR,  SAR  and 
SIGINT  sensor  requirements.  The  RQ-4  can  satisfy  13  EO/IR,  18  SAR,  and  17  SIGINT 
requirements.  The  RQ-1  can  provide  SAR  coverage  of  the  entire  COI  with  the  exception 
of  a  small  area  in  the  southwest  comer  and  SIGINT  coverage  of  the  whole  country. 


Figure  37.  RQ-4  Sensor  Coverage  for  the  Penetrating  Scenario 

The  ability  of  the  P-3  to  provide  EO/IR  and  SAR  coverage  is  illustrated  in 
Figure  38.  The  P-3  can  range  13  EO/IR  missions  and  18  SAR  missions.  Like  the  RQ-1 
and  RQ-4,  the  P-3  can  range  all  but  a  small  portion  of  the  country  with  its  SAR  sensor. 
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Figure  38.  P-3  Sensor  Coverage  for  the  Penetrating  Scenario 

Figure  39  depicts  the  coverage  of  the  RC-135.  The  RC-135  has  the  ability 
to  meet  all  17  SIGINT  sensor  requirements  and  can  cover  the  entire  country. 


Figure  39.  RC-135  Sensor  Coverage  for  the  Penetrating  Scenario 

The  U-2’s  sensor  capabilities  are  shown  in  Figure  40.  The  U-2  can  satisfy 
17  EO/IR,  18  SAR,  and  17  SIGINT  missions.  The  SAR  and  SIGINT  sensor  packages  on 
the  U-2  can  range  any  location  in  the  COI.  Just  as  in  the  non-penetrating  scenario,  the  U- 
2  is  the  most  capable  platform,  sensor  wise. 
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U-2  Sensor  Coverage 


EO/IR  SAR  SIGINT 


Figure  40.  U-2  Sensor  Coverage  for  the  Penetrating  Scenario 

Table  1 1  provides  a  summary  of  the  number  of  missions  that  can  be 
covered  by  each  ISR  platform  from  each  mission  area.  Ranked  by  ability  to  provide 
coverage  of  the  most  targets  per  mission  area,  the  most  capable  aircraft  is  the  U-2, 
followed  by  the  RQ-4,  the  RC-135,  the  P-3,  and  finally  the  RQ-1.  This  further 
demonstrates  the  U-2’s  superior  ability  to  provide  coverage.  Note  that  the  capability  of 
the  RQ-1  and  the  P-3  is  very  similar,  with  the  P-3  marginally  exceeding  the  RQ-1. 


MISSION 

AREA 

RQ-4 

RQ-1 

P-3 

RC-135 

U-2 

EO/IR 

SAR 

SIGINT 

TOTAL 

EO/IR 

SAR 

TOTAL 

EO/IR 

SAR 

TOTAL 

SIGINT/TOTAL 

EO/IR 

SAR 

SIGINT 

TOTAL 

1 

1 

1 

5 

7 

0 

1 

1 

1 

1 

2 

5 

1 

2 

6 

9 

2 

0 

2 

3 

5 

0 

2 

2 

0 

2 

2 

3 

1 

2 

6 

9 

3 

0 

1 

3 

4 

0 

1 

1 

0 

1 

1 

3 

1 

1 

5 

7 

4 

0 

1 

4 

5 

0 

1 

1 

0 

1 

1 

4 

0 

2 

7 

9 

5 

0 

2 

5 

7 

0 

2 

2 

0 

2 

2 

5 

0 

3 

8 

11 

6 

1 

2 

4 

7 

0 

2 

2 

1 

2 

3 

4 

1 

3 

6 

10 

7 

0 

3 

2 

5 

0 

3 

3 

0 

3 

3 

2 

1 

3 

6 

10 

8 

0 

3 

4 

7 

0 

3 

3 

0 

3 

3 

4 

1 

4 

6 

11 

9 

0 

3 

2 

5 

0 

3 

3 

0 

3 

3 

2 

1 

4 

5 

10 

10 

0 

2 

1 

3 

0 

2 

2 

0 

2 

2 

1 

1 

2 

4 

7 

11 

0 

1 

1 

2 

0 

1 

1 

0 

1 

1 

1 

0 

1 

2 

3 

12 

0 

1 

1 

2 

0 

1 

1 

0 

1 

1 

1 

0 

3 

2 

5 

13 

0 

2 

1 

3 

0 

2 

2 

0 

2 

2 

1 

1 

2 

1 

4 

24 

0 

1 

3 

4 

0 

1 

1 

0 

1 

1 

3 

0 

1 

6 

7 

51 

1 

2 

6 

9 

1 

2 

3 

1 

2 

3 

6 

1 

4 

8 

13 

52 

1 

1 

5 

7 

0 

1 

1 

1 

1 

2 

5 

1 

2 

9 

12 

53 

2 

3 

8 

13 

0 

3 

3 

2 

3 

5 

8 

2 

5 

10 

17 

54 

1 

4 

8 

13 

1 

4 

5 

1 

4 

5 

8 

1 

4 

11 

16 

55 

1 

0 

3 

4 

1 

0 

i 

1 

0 

i 

3 

1 

0 

4 

5 

56 

0 

3 

5 

8 

0 

3 

3 

0 

3 

3 

5 

0 

3 

8 

ii 

57 

1 

4 

8 

13 

0 

4 

4 

1 

4 

5 

8 

1 

7 

10 

18 

58 

1 

3 

6 

10 

0 

3 

3 

1 

3 

4 

6 

1 

5 

6 

12 

59 

1 

3 

3 

7 

0 

3 

3 

1 

3 

4 

3 

1 

5 

8 

14 

60 

0 

2 

4 

6 

0 

2 

2 

0 

2 

2 

4 

1 

2 

5 

8 

61 

1 

4 

6 

11 

0 

4 

4 

1 

4 

5 

6 

2 

5 

9 

16 

62 

2 

2 

1 

5 

0 

2 

2 

2 

2 

4 

1 

2 

2 

4 

8 

TOTAL 

14 

56 

102 

172 

3 

56 

59 

14 

56 

70 

102 

23 

77 

162 

262 

MIN 

0 

0 

1 

3 

0 

0 

1 

0 

0 

1 

1 

0 

0 

1 

4 

MAX 

2 

4 

8 

13 

1 

4 

5 

2 

4 

5 

8 

2 

7 

11 

18 

MEAN 

0.9 

2.4 

4.8 

8.1 

0.2 

2.4 

2.6 

0.9 

2.4 

3.3 

4.8 

1.1 

3.4 

7.1 

11.5 

MEDIAN 

1 

2.5 

5 

7.5 

0 

2.5 

3 

1 

2.5 

3.5 

5 

1 

3.5 

8 

12 

Table  11.  Penetrating  Scenario:  Number  of  Missions  Potentially  Satisfied  from  Each 
Mission  Area  by  Platfonn  and  Sensor  Type 
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Figure  41  compares  the  ability  of  the  various  ISR  platforms  to  provide 
coverage.  Note  that  the  U-2  is  superior  to  all  other  aircraft  in  every  category.  This 
difference  is  due  to  the  U-2’s  advantage  in  sensor  range  for  all  types  of  sensors. 


Figure  4 1 .  Potential  Mission  Coverage  by  ISR  Platform 


Figure  42  illustrates  the  predicted  time  on  station  for  each  ISR  platform 
from  each  base.  The  RQ-4  with  its  comparatively  high  speed  and  36  hour  mission 
duration  is  the  clear  leader  in  the  ability  to  remain  on-station.  Interestingly,  the  RQ-1 
provides  the  least  amount  of  time  on-station  per  sortie.  Despite  a  24  hour  mission 
duration,  the  RQ-4’s  on-station  endurance  is  limited  by  its  slow  speed.  The  percentage  of 
time  on-station  increases  slightly  for  platforms  at  Bases  A  and  D  and  decreases  slightly 
for  aircraft  at  Bases  B  and  C  when  compared  to  the  non-penetrating  scenario  due  to  the 
removal  of  waypoint  routing.  As  in  the  non-penetrating  scenario,  it  is  anticipated  that  the 
ability  to  cover  a  greater  number  of  targets  per  sortie  will  be  more  important  to  overall 
coverage  than  the  ability  to  remain  on-station  for  a  longer  period  of  time. 
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Predicted  Percentage  of  Time  On-Station  by  Base  and  Platform  Type 


Figure  42.  Predicted  Percentage  of  Time  On-Station  by  Base  and  Platform  Type 


The  ability  to  satisfy  all  of  the  missions  in  the  scenario  is  shown  in  Table 
12.  There  are  17  of  20  EO/IR,  18  of  18  SAR,  and  17  of  17  SIGINT  requirements  that  can 
be  met.  This  indicates  that  approximately  95%  of  the  sensor  requirements  in  the  scenario 
can  be  satisfied  by  the  available  platforms.  This  does  not  mean,  however,  that  95%  is  the 
amount  of  coverage  that  will  be  achieved  in  a  simulation.  The  effects  of  transit  time, 
attrition,  and  platform  on-station  time,  among  other  factors  will  keep  the  total  coverage 
from  reaching  95%. 
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EO/IR  Sensor  Requirements 

Mission 

RQ-4 

RQ-1 

P-3 

RC-135  U-2  Any 

1.1 

***  NO  SENSOR  REQUIREMENT 

*** 

2.1 

1 

0 

1 

0 

1 

3 
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1 

1 

1 

0 

1 

4 
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2 

1 
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2 

7 
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0 

0 

0 

0 

°l 
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0 

0 
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0 

2 

2 
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***  NO  SENSOR  REQUIREMENT 
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0 

0 

0 

0 

2 
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1 
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1 
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1 
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1 
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1 
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1 

0 

1 

0 

1 
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1 

0 

1 

0 

1 

3 
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1 

0 

1 

0 

3 

5 
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0 

0 

0 

0 

1 

1 
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1 

0 

1 

0 

1 

3 

19.1 

0 

0 

0 

0 

°l 
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1 

0 

1 

0 

1 

3 

21.1 

1 

0 

1 

0 

2 

4 

22.1 

1 

1 

1 

0 

1 

4 

23.1 

***  NO  SENSOR  REQUIREMENT 

*** 

24.1 

1 

0 

1 

0 

1 

3 
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0 

0 

0 

0 

°l 

SAR  Sensor  Requirements 

SIGINT  Sensor  Requirements 

Mission 

RQ-4 

RQ-1 

P-3 
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0 

4 

5 
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1 
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1 

0 

1  4 

2.3 
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0 
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6 

7 

19 
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4 

4 

4 

0 
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4.3 

5 

0 

0 

5 

9 

19 
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2 

2 

2 

0 

4  10 

5.3 

7 

0 

0 

7 

12 

26 

6.1 

3 

3 

3 

0 

5  14 

6.3 

7 

0 

0 

7 

9 

23 

7.1 

***  NO  SENSOR  REQUIREMENT  *** 

7.3 

8 

0 

0 

8 

12 

28 

8.1 

***  NO  SENSOR  REQUIREMENT  *** 

8.3 

3 

0 

0 

3 

6 

12 

9.1 

4 

4 

4 

0 

4  16 

9.3 

7 

0 

0 

7 

11 

25 

10.1 

3 

3 

3 

0 

5  14 

10.3 

7 

0 

0 

7 

14 

28 

11.1 

***  NO  SENSOR  REQUIREMENT  *** 

11.3 

7 

0 

0 

7 

14 

28 

12.1 

2 

2 

2 

0 

6  12 

12.3 

***  NO  SENSOR  REQUIREMENT 

*** 

13.1 

3 

3 

3 

0 

3  12 

13.3 

***  NO  SENSOR  REQUIREMENT 

*** 

14.1 

4 

4 

4 

0 

6  18 

14.3 

8 

0 

0 

8 

10 

26 

15.1 

2 

2 

2 

0 

4  10 

15.3 

7 

0 

0 

7 

14 

28 

16.1 

4 

4 

4 

0 

5  17 

16.3 

6 

0 

0 

6 

10 

22 

17.1 

4 

4 

4 

0 

5  17 

17.3 

***  NO  SENSOR  REQUIREMENT 

*** 

18.1 

***  NO  SENSOR  REQUIREMENT  *** 

18.3 

***  NO  SENSOR  REQUIREMENT 

*** 

19.1 

5 

5 

5 

0 

6  21 

19.3 

***  NO  SENSOR  REQUIREMENT 

*** 

20.1 

3 

3 

3 

0 

5  14 

20.3 

***  NO  SENSOR  REQUIREMENT 

*** 

21.1 

2 

2 

2 

0 

3  9 

21.3 

4 

0 

0 

4 

8 

16 

22.1 

3 

3 

3 

0 

3  12 

22.3 

***  NO  SENSOR  REQUIREMENT 

*** 

23.1 

***  NO  SENSOR  REQUIREMENT  *** 

23.3 

2 

0 

0 

2 

4 

8 

24.1 

3 

3 

3 

0 

4  13 

24.3 

6 

0 

0 

6 

9 

21 

25.1 

4 

4 

4 

0 

4  16 

25.3 

8 

0 

0 

8 

8 

24 

Table  12.  Penetrating  Scenario:  Ability  to  Satisfy  Sensor  Requirements  by  Platform 
Type  (Cells  highlighted  in  red  indicate  that  the  sensor  requirement  cannot  be 
met  by  any  platform  type) 


c.  Assessment 


The  following  assessments  are  made  based  on  the  preceding  analysis  of 
the  factors  affecting  the  penetrating  scenario: 

•  The  U-2’s  ability  to  cover  the  most  missions  per  sortie  coupled  with  its 
relatively  high  speed  will  make  it  the  most  significant  asset  in  the 
simulation. 

•  The  RQ-4  will  be  important  in  maximizing  coverage  due  to  its  long 
mission  duration  and  relatively  strong  sensor  package. 

•  The  relatively  long  range  of  the  SIGINT  sensor  and  the  second  best  on- 
station  performance  may  make  the  RC-135  a  significant  contributor  to 
overall  coverage. 

•  The  P-3’s  relatively  short  sensor  ranges  and  the  lack  of  a  SIGINT  sensor 
will  limit  its  contribution  to  mission  coverage. 

•  It  is  expected  that  the  minimal  ability  to  cover  EO/IR  targets,  along  with 
the  lack  of  a  SIGINT  sensor,  will  limit  the  utilization  of  the  RQ-1  within 
the  simulation. 

•  The  fact  that  95%  of  the  sensor  requirements  can  be  met  by  the  available 
platforms  will  dramatically  improve  coverage  compared  to  the  non¬ 
penetrating  scenario. 
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C.  MEASURES  OF  EFFECTIVENESS  (MOE) 

The  JDAFS  simulation  provides  for  the  generation  of  a  number  of  MOEs 
including,  Coverage,  Coverage  by  Type,  and  Attrition. 

1.  Non-Penetrating  Scenario  Coverage 
a.  Coverage  Regression  Model 

An  initial  plot  of  the  distribution  and  summary  statistics  for  274  design 
points  in  the  non-penetrating  scenario  is  shown  in  Figure  43.  The  coverage  results  appear 
to  be  roughly  normally  distributed  with  a  mean  of  0.48.  The  five  outlying  data  points  at 
the  bottom  of  the  outlier  box  plot  warrant  further  examination. 


Coverage 


0.53-^ 


0.51 

0.49 

0.47 

0.45 

0.43+1 

0.41- 

0.39- . 


Quantiles 


Moments 


100.0%  maximum  0.53257 
99.5%  0.53218 

97.5%  0.52125 

90.0%  0.50897 

75.0%  quartile  0.49836 

50.0%  median  0.48424 

25.0%  quartile  0.47352 

10.0%  0.46223 

2.5%  0.44862 

0.5%  0.40572 

0.0%  minimum  0.39613 


Mean 
Std  Dev 
Std  Err  Mean 
upper  95%  Mean 
lower  95%  Mean 
N 


0.4848123 

0.0193636 

0.0011698 

0.4871153 

0.4825093 

274 


Figure  43.  Distribution  and  Summary  Statistics  of  Mission  Area  Coverage 


An  examination  of  the  data  reveals  that  four  of  the  outlying  data  points  1 , 
2,  4,  and  265  are  the  result  of  design  points  having  only  a  single  base  available  for 
operations  (See  Table  13).  Even  though  each  operating  location  had  a  full  compliment  of 
aircraft  (21  total  airframes),  the  mean  coverage  ranked  at  the  bottom  of  all  the  results. 
When  ranked  from  greatest  to  least  coverage,  D,  C,  A,  then  B,  this  ordering  is  not 
surprising  when  compared  to  the  mean  distance  of  the  bases  to  their  mission  areas  in  the 
pre-simulation  analysis.  Having  a  single  operating  base  available  to  ISR  platforms  in  a 
region  of  the  size  encompassed  in  this  study  is  unrealistic.  Therefore,  the  design  points 
that  include  only  one  operating  location  will  not  be  considered  for  the  remainder  of  this 
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analysis.  The  remaining  outlier,  data  point  70,  had  ISR  platforms  at  all  locations  (30  total 
airframes);  however,  having  only  one  U-2  and  a  near  maximum  optimization  interval 
appear  to  be  the  reason  for  the  low  mean  coverage.  There  is  no  easy  or  obvious 
justification  for  removing  this  design  point  from  the  data  set  for  further  analysis. 
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RC-135 

U-2 

RQ-4 

RQ-1 
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0 

0 
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0 

0 

0 

0 

0 

0 

0 

3 

6 

6 

3 

3 
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2 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

3 

6 

6 

3 

3 
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0 

0 

0 

0 
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0 
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0 
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6 

3 

3 

0 

0 
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0 

0 

0 

0 

0 

0 

0 
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1 

2 

0 

1 

0 

3 

3 

2 

1 

1 

0 

4 

3 

1 

0 

1 

4 

3 

0 

0 
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0.4267 

265 

3 

6 

6 

3 

3 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

3.25 

0.4217 

Table  13.  Non-Penetrating  Scenario:  Low  Outlying  Data  Points 


For  comparison  with  future  models,  a  full  quadratic  model  with  the 
outlying  data  points  was  constructed.  Figure  44  shows  this  model.  Note  that  the  R- 
squared16  value  for  this  model  is  0.73. 

Actual  by  Predicted  Plot 


n — i — i — i — i — i — i — i — i — i — 

.39  .41  .43  .45  .47  .49  .51  .53 
Coverage  Predicted 
Pc.OOOl  RSq=0.73  RMSE=0.0105 


Figure  44.  Actual  by  Predicted  Coverage  Plot  for  Full  Quadratic  Model  with  Single 
Base  Outliers  Included. 


After  removing  the  single  base  outliers,  the  distribution  and  summary 
statistics  for  the  non-penetrating  scenario  data  were  recalculated.  The  increase  in  the 
mean  coverage  is  negligible,  but  the  remaining  data  more  closely  approximates  a  normal 
distribution  (See  Figure  45). 


R~  or  R-squared,  the  coefficient  of  determination,  is  defined  as  the  proportion  of  response 
variation  that  is  explained  by  the  regression  model  (Devore  2004,  514). 
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Coverage 


Quantiles 


Moments 


100.0% 

maximum 

0.53257 

Mean 

0.485819 

99.5% 

0.53220 

Std  Dev 

0.0175688 
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0.52137 

Std  Err  Mean 

0.0010692 

90.0% 
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0.487924 

75.0% 
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lower  95%  Mean 

0.4837139 
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N 
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0.47372 

10.0% 

0.46318 

2.5% 

0.45283 

0.5% 

0.43413 

0.0% 
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0.42671 

Figure  45.  Non-Penetrating  Scenario:  Distribution  and  Summary  Statistics  of  Mission 
Area  Coverage  ( single  base  outliers  removed) 


From  the  final  270  design  point  dataset  a  full  quadratic  model  with  main 
effect,  interaction,  and  polynomial  terms  was  created.  The  Stepwise  Regression  Control 
settings  within  JMP  were  as  follows: 

•  Probability  to  enter:  0.01 

•  Probability  to  leave:  0.01 

•  Direction:  Mixed 

The  Construct  Model  Effects  macros  for  Factorial  to  Degree  and  Polynomial  to  Degree 
were  set  at  2  to  allow  two-way  interaction  and  quadratic  terms. 


The  resultant  model  Achieves  an  R-squared  of  0.78  and  contains  12  main 
effect  terms,  4  interaction  terms,  and  3  second  order  terms.  See  Figures  46  and  47  for  the 
regression  plot  and  a  list  of  the  regression  model  terms. 


Actual  by  Predicted  Plot 
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Figure  46.  Actual  by  Predicted  Coverage  Plot  for  Full  Quadratic  Model  with  Single 
Base  Outliers  Removed 
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Parameter  Estimates 

Term 

Estimate 

Std  Error 

t  Ratio 

Prot»|t| 

Intercept 

0.4762323 

0.002843 

167.51 

<.0001* 

A  RQ-4 

-0.002767 

0.000534 

-5.18 

<.0001* 

A  RQ-1 

-0.000885 

0.000285 

-3.10 

0.0021* 

A  RC-135 

0.0039884 

0.000535 

7.45 

<.0001* 

A  U-2 

0.0056309 

0.000536 

10.50 

<.0001* 

B  RQ-4 

-0.002767 

0.000541 

-5.12 

<.0001* 

B  RC-135 

-0.000835 

0.000538 

-1.55 

0.1219 

B  U-2 

0.0035851 

0.000534 

6.71 

<.0001* 

C  RQ-4 

-0.002511 

0.000531 

-4.73 

<.0001* 

C  U-2 

0.0058404 

0.000531 

10.99 

<.0001* 

D  RQ-4 

-0.001957 

0.000529 

-3.70 

0.0003* 

D  U-2 

0.0074843 

0.000532 

14.07 

<.0001* 

OPT  INT 

-0.004388 

0.000361 

-12.15 

<.0001* 

(A  RQ-4-1. 52963 )*(B  RC-1 35-1. 52963) 

-0.002434 

0.000522 

-4.66 

<.0001* 

(A  RC-1 35-1. 52963)*(D  RQ-4-1 .52963) 

-0.001769 

0.000518 

-3.41 

0.0007* 

(B  RC-1 35-1. 52963)*(D  RQ-4-1 .52963) 

-0.001793 

0.000535 

-3.35 

0.0009* 

(C  RQ-4-1. 52963)*(D  U-2-1. 52963) 

-0.002549 

0.000504 

-5.05 

<.0001* 

(B  RQ-4-1. 52963)*(B  RQ-4-1 .52963) 

0.001986 

0.000574 

3.46 

0.0006* 

(C  U-2-1. 52963)*(C  U-2-1. 52963) 

-0.001624 

0.00057 

-2.85 

0.0047* 

(D  RQ-4-1. 52963)*(D  RQ-4-1 .52963) 

0.0030399 

0.000559 

5.44 

<.0001* 

Figure  47.  Full  Quadratic  Coverage  Model  Terms 


As  expected,  the  most  capable  platforms,  the  U-2  and  RQ-4,  show  up  as 
terms  from  each  base.  The  RC-135  is  reflected  twice  from  Bases  A  and  B.  The  RQ-1 
from  Base  A  is  the  only  entry  for  that  platform.  Interestingly,  the  P-3  is  not  represented 
in  the  model  at  all.  In  addition  to  the  aircraft  factors,  the  optimization  interval  term  is 
also  included  in  the  final  model. 

To  test  the  validity  of  the  Coverage  Full  Quadratic  Regression  model,  the 
regression  assumptions  from  Part  A  of  this  chapter  must  be  verified.  The  Actual  by 
Predicted  Plot  in  Figure  46  indicates  that  the  relationship  between  the  response  variable 
(coverage)  and  the  regressors  is  reasonably  linear.  The  apparently  random  cloud  of  data 
points  with  a  mean  of  zero  in  the  Residual  by  Predicted  Plot,  Figure  48,  confirms  that  the 
requirements  for  a  zero  mean,  constant  variance,  and  lack  of  correlation  of  the  residuals  is 
met.  Compliance  with  the  normality  assumption  is  shown  in  Figure  49  by  a  histogram  of 
the  residuals  overlaid  with  a  normal  curve  and  a  normal  quantile  plot. 
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Figure  48.  Residual  by  Predicted  Plot  for  Coverage  Model 


Figure  49.  Flistogram  and  Normal  Quantile  Plot  of  Residuals  for  Coverage  Model 
b.  Coverage  Factor  Interactions 

To  further  examine  the  interactions  of  the  factors  in  the  model,  an 

Interaction  Profile  Plot,  Figure  50,  was  constructed.  Each  of  the  individual  cells  within 

the  larger  plot  display  the  interaction  between  two  factors  and  their  effect  on  Coverage. 

Each  cell  contains  the  plot  of  two  lines,  one  for  the  high  setting  and  one  for  the  low 

setting  of  the  term  found  in  the  name  box  for  that  row.  Solid  lines  or  curves  indicate  the 

presence  of  interaction  between  terms.  The  lighter,  broken  lines  or  curves  indicate  no 

interaction.  Note  that  the  lines  without  interaction  are  nearly  parallel.  The  y  axis  of  the 

grid  is  the  response  variable,  coverage,  and  the  x  axis  indicates  the  levels  for  each  factor. 

When  assessing  the  change  to  the  response  variable  based  on  moving  along  one  of  the 

lines  in  a  single  cell,  it  is  assumed  that  all  other  factors  are  held  constant. 
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Interaction  Profiles 
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Figure  50.  Interaction  Profile  for  Final  Coverage  Model 


The  following  figures  displaying  terms  with  significant  interaction  are 
taken  from  the  larger  Interaction  Profile  plot,  Figure  50.  Note  that  in  all  cases,  the 
coverage  varies  only  marginally  regardless  of  the  settings  of  the  interaction  terms  for  this 
model. 

•  Figure  51  shows  that  with  no  RQ-4s  at  Base  A,  total  coverage  actually 
increases  with  the  addition  of  RC- 135s  at  Base  B.  However,  if  there  are  3 
RQ-4s  at  Base  A,  coverage  declines  as  RC-135s  are  added  at  Base  B. 
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Figure  5 1 .  Interaction  Profile  for  A  RQ-4  and  B  RC-135 

•  Figure  52  shows  that  the  RC-135s  at  Base  A  have  a  quadratic  interaction 
with  the  addition  of  RQ-4s  at  Base  D.  That  is,  regardless  of  the  number  of 
RC-135s  at  Base  A,  coverage  initially  declines  and  then  rises  with  the 
addition  of  RQ-4s  at  Base  D. 


Figure  52.  Interaction  Profile  for  A  RC-135  and  D  RQ-4 

•  Figure  53  shows  that  with  no  RC-135s  at  Base  B,  total  coverage  actually 
increases  slightly  with  the  addition  of  RQ-4s  at  Base  A.  Flowever,  if  there 
are  3  RC-135s  at  Base  B,  coverage  declines  as  RQ-4s  are  added  at  Base  A. 


Figure  53.  Interaction  Profile  for  B  RC-135  and  A  RQ-4 


Figure  54  shows  that  the  RC-135s  at  Base  B  have  a  quadratic  interaction 
with  the  addition  of  RQ-4s  at  Base  D.  That  is,  regardless  of  the  number  of 
RC-135s  at  Base  B,  coverage  initially  declines  and  then  rises  with  the 
addition  of  RQ-4s  at  Base  D. 
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Figure  54.  Interaction  Profile  for  B  RC-135  and  D  RQ-4 

•  Figure  55  shows  that  with  the  number  of  RQ-4s  at  Base  C  at  zero  or  three, 
total  coverage  increases  with  the  addition  of  U-2s  at  Base  D.  The  slope  of 
the  line  for  zero  RQ-4s  is  steeper,  indicating  that  it  is  better  to  have  no 
RQ-4’s  at  Base  C  than  to  have  the  maximum  number. 


Figure  55.  Interaction  Profile  for  C  RQ-4  and  DU-2 

•  Figure  56  shows  that  with  the  number  of  RQ-4s  at  Base  D  at  zero  or  three, 
total  coverage  increases  with  the  addition  of  RC-135s  at  Base  A.  The 
slope  of  the  line  for  zero  RQ-4s  is  steeper,  indicating  that  it  is  better  to 
have  no  RQ-4’s  at  Base  D  than  to  have  the  maximum  number. 


Figure  56.  Interaction  Profile  for  D  RQ-4  and  A  RC-135 
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•  Figure  57  shows  that  with  no  RQ-4s  at  Base  D,  total  coverage  actually 
increases  with  the  addition  of  RC-135s  at  base  B.  However,  if  there  are  3 
RQ-4s  at  Base  D,  coverage  declines  as  RC-135s  are  added  at  Base  B. 


Figure  57.  Interaction  Profile  for  D  RQ-4  and  B  RC-135 

•  Figure  58  shows  that  with  no  U-2s  at  Base  D,  total  coverage  increases 
slightly  with  the  addition  of  RQ-4s  at  Base  C.  However,  if  there  are  3  U- 
2s  at  Base  D,  coverage  declines  as  RQ-4s  are  added  at  Base  C. 


Figure  58.  Interaction  Profile  for  D  U-2  and  C  RQ-4 

In  several  of  the  cases  illustrated  above,  coverage  actually  declines  with 
addition  of  more  aircraft.  This  is  counter  intuitive.  Normally,  the  expectation  is  that 
more  assets  provide  more  coverage.  The  seeming  anomaly  is  explained  by  the 
optimization  that  takes  place  within  JDAFS.  The  optimization  algorithm  finds  the  best 
coverage  possible  based  on  matching  the  assets  available  to  the  missions  to  be  serviced. 
In  the  simulation,  only  one  platform  can  be  assigned  to  a  mission.  Once  that  assignment 
is  made  no  additional  aircraft  can  be  assigned  to  the  same  missions.  The  addition  of 
more  aircraft  can  result  in  inferior  platforms  being  assigned  to  missions,  thereby 
preventing  more  capable  aircraft  from  having  the  opportunity  for  maximum  assignment. 
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c.  Coverage  Regression  Tree  Analysis 

Another  means  of  exploring  the  data  and  confirming  the  validity  of  the 
regression  model  is  through  the  use  of  classification  and  regression  trees  (CART).  To 
determine  the  number  of  splits  that  would  continue  to  yield  additional  explanatory  value 
for  Coverage,  20  splits  were  done  in  rapid  succession  to  record  the  R-squared  value 
without  regard  to  the  content  or  structure  of  the  regression  tree.  These  R-squared  values 
were  plotted  to  find  the  point  that  the  curve  began  to  plateau,  indicating  diminishing 
returns  from  each  successive  split.  In  this  case,  1 1  splits  are  warranted.  Figure  59 
provides  a  plot  of  the  R-squared  values  against  the  number  of  splits. 
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R-Squared  vs.  Number  of  Splits 


0.7 


.  • 

• 

.  •  * 

•>  * 

. N 

• 

• 

• 

• 

• 

• 

• 

• 

0.529 

0.5 


o'  0.3 


0.0 


6  8  10  11  12  14 

Number  of  Regression  Tree  Splits 


Figure  59.  R-Squared  vs.  Number  of  Regression  Tree  Splits 


Figure  60  shows  the  regression  tree  for  Coverage  with  1 1  splits.  The  full 
tree  and  partition  graph  can  be  found  in  Appendix  I.  The  first  split  indicates  that  better 
coverage  is  obtained  when  the  optimization  interval  is  4.64  hours  or  less.  The  second  and 
third  splits  are  on  the  presence  of  U-2s.  In  the  second  split,  coverage  is  improved  by 
having  two  or  more  U-2s  at  Base  D  and  in  the  third  split  coverage  is  improved  by  having 
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at  least  one  U-2  at  Base  C.  The  fourth  split  indicates  that  better  coverage  is  gained  by 
having  at  least  one  RQ-4  at  Base  B.  The  fifth  split  continues  to  demonstrate  the 
importance  of  the  U-2,  showing  that  it  is  beneficial  to  have  two  or  more  U-2s  at  Base  A. 
The  sixth  split  indicates  that  coverage  is  improved  by  the  stationing  at  least  one  RC-135 
at  Base  A.  The  seventh  split  indicates  that  no  RQ-4  platforms  at  Base  A  improves 
coverage.  The  eighth  split  is  a  return  to  the  optimization  interval,  showing  that  coverage 
improves  with  optimization  intervals  of  3.03  hours  or  less.  The  most  significant  factor  at 
the  ninth  split  is  U-2s  at  Base  A,  indicting  that  coverage  is  improved  by  having  at  least 
one  U-2  at  Base  A.  Split  ten  indicates  improved  coverage  by  stationing  one  or  more  RC- 
135s  at  Base  A.  The  final  split  again  is  on  U-2s  at  Base  A,  this  time  indicating  improved 
coverage  with  one  or  more  U-2s  at  Base  A. 

The  splits  in  the  regression  tree,  especially  the  multiple  splits  on 
optimization  interval,  Base  A  U-2s,  and  the  split  on  Base  D  U-2s,  serve  to  reinforce  the 
findings  from  the  regression  model. 
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Figure  60.  Regression  Tree  for  Coverage 
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d.  Coverage  Contour  Plot  Analysis 

To  further  explore  the  significance  of  the  U-2  as  a  primary  driver  of  the 
level  of  Coverage  in  the  non-penetrating  regression  model,  a  contour  plot  was  generated 
to  illustrate  the  how  the  number  of  U-2s  relates  to  the  total  number  of  aircraft.  The 
various  color  filled  regions  of  the  plot  represent  the  amount  of  coverage  attained  as  the 
number  of  U-2s  and  total  aircraft  vary.  The  combinatorial  nature  of  the  problem  and 
strong  interactions  of  the  regressors,  coupled  with  the  fact  that  the  other  factors  are  not 
held  fixed,  results  in  a  non-smooth  contour  plot.  Additionally,  the  combinatorial  nature 
and  base  locations  along  with  the  approximate  dynamic  programming  technique  cause 
“islands”  in  the  contour  plot.  The  contour  plot  is  read  by  selecting  a  value  on  the  x-axis, 
TOTAL  ACFT,  and  then  examining  the  corresponding  coverage  based  on  number  of  U-2 
platforms  on  the  y-axis  (i.e.,  given  30  total  aircraft,  as  the  number  of  U-2  platforms 
ranges  between  1  and  9,  coverage  ranges  from  <  48.3%  to  >  50%).  Figure  61 
demonstrates  that  in  nearly  all  cases,  coverage  is  improved  by  increasing  the  number  of 
U-2  platforms.  It  appears  that  having  three  or  less  U-2’s  dramatically  limits  the  level  of 
coverage  while  six  or  more  tends  to  drive  coverage  up.  The  value  of  additional  U-2 
assets  peaks  at  1 1 . 


Contour  Plot  for  Coverage 


12- 

10- 

8- 

CM 

=>  6- 


- t-1 — t-1-! — r-r-' — n — i  '  i 

10  20  30  40  50  60  70  80  9C 

TOTAL  ACFT 


Legend  - 


Mean(coverage) 
™  <=  0.438 
™  <=  0.450 
<=  0.463 
™  <=  0.475 
™  <=  0.488 
™  <=  0.500 
>  0.500 


Figure  61 .  Coverage  Contour  Plot  of  Coverage  for  U-2  vs.  Total  Aircraft 
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2.  Non-Penetrating  Scenario  Coverage  by  Type 

The  coverage  by  type  MOE  is  a  subset  of  total  coverage.  Recall  that  mean 
coverage  for  the  non-penetrating  scenario  was  48.6%.  Of  this  total  amount  of  coverage, 
the  RQ-4  is  the  largest  contributor  at  38.2%  of  the  total,  slightly  ahead  of  the  U-2  at 
36.9%.  These  two  platforms  alone  account  for  over  75%  of  the  total  coverage.  This  is 
not  unexpected  given  the  superior  sensor  packages  carried  by  these  airframes.  It 
appeared  in  pre- simulation  analysis  that  the  RQ-1  would  not  be  a  primary  contributor  to 
total  coverage  given  its  relatively  limited  sensor  range.  This  is  clearly  not  the  case.  The 
long  dwell  time  apparently  gives  the  RQ-1  a  significant  advantage  over  the  P-3  and  the 
RC-135.  The  RC-135s  total  contribution  is  likely  limited  since  it  carries  only  the 
SIGINT  sensor.  This  disadvantage  is  compounded  by  the  fact  that  there  are  fewer 
SIGINT  targets  in  the  scenario  than  EO/IR  or  SAR  targets.  See  Figure  62  for  a 
comparison  of  coverage  by  platform  type. 


Coverage  by  Platform  Type 
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Figure  62.  Coverage  by  Platform  Type 
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3.  Penetrating  Scenario  Coverage 

a.  Coverage  Regression  Model 

An  initial  plot  of  the  distribution  and  summary  statistics  for  274  design 
points  in  the  penetrating  scenario  is  shown  in  Figure  63.  The  coverage  results  appear  to 
be  roughly  normally  distributed  with  a  mean  of  0.48.  The  seven  low  outlying  and  four 
high  outlying  data  points  in  the  outlier  box  plot  warrant  examination. 
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0.5962119 

75.0% 

quartile 

0.62264 

ower  95%  Mean 

0.5798841 

50.0% 

median 

0.59131 

N 

274 

Coverage 


25.0% 

10.0% 

2.5% 

0.5% 

0.0% 


quartile 


minimum 


0.55980 

0.51609 

0.46004 

0.22126 

0.22036 


Figure  63.  Penetrating  Scenario:  Distribution  and  Summary  Statistics  of  Mission 
Area  Coverage 


An  examination  of  the  data  reveals  that  four  of  the  low  outlying  data 
points  1,  2,  4,  and  265  are  the  result  of  design  points  having  only  a  single  base  available 
for  operations  (See  Table  14).  Even  though  each  operating  location  had  a  full 
compliment  of  aircraft  (21  total  airframes),  the  mean  coverage  ranked  at  the  bottom  of  all 
the  results.  Having  a  single  operating  base  available  to  ISR  platforms  in  a  region  of  the 
size  encompassed  in  this  study  is  unrealistic.  Therefore,  the  design  points  that  include 
only  one  operating  location  are  not  considered  in  the  remained  of  this  analysis.  The 
remaining  outliers,  data  points  11,  38,  and  27,  had  ISR  platforms  at  all  locations  (17,  33, 
and  30  total  airframes,  respectively,  out  of  a  possible  84).  The  low  total  numbers  of 
aircraft,  coupled  with  low  numbers  of  U-2  (3,  6,  and  3,  respectively)  and  RQ-4  (2,  2,  and 
3,  respectively)  airframes  appears  to  be  the  reason  for  the  low  mean  coverage.  There  is 
no  justification  for  removing  these  design  points  from  the  data  set  in  further  analysis. 
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Design 

Point 
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RQ-1 

/T 
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A 
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nr 
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B 

RQ-4 

B 
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B 
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U-2 
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nr 
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c 
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nr 
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D 
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nr 
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D 
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nr 

U-2 
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1 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

3 

6 

6 

3 

3 

3.25 
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2 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

3 

6 

6 

3 

3 

0 

0 

0 

0 

0 

3.25 
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4 

0 

0 

0 

0 

0 

3 

6 

6 

3 

3 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

3.25 
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11 

0 

1 

0 

1 

0 

0 

1 

2 

0 

1 

1 

1 

2 

0 

1 

1 

2 

1 

1 

1 

0.98 
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28 

0 

3 

2 

3 

2 

0 

5 

1 

1 

1 

0 

4 

0 

2 

2 

2 

1 

3 

0 

1 

3.91 
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37 

0 

4 

4 

2 

2 

2 

5 

0 

0 

1 

0 

2 

1 

3 

0 

1 

0 

0 

3 

0 

1.45 

0.440144 
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3 

6 

6 

3 

3 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

3.25 

0.263309 

Table  14.  Penetrating  Scenario:  Low  Outlying  Data  Points 


Four  high  outlier  data  points  were  also  identified,  86,  261,  273  and  274 
(See  Table  15).  Each  of  these  points  contains  high  numbers  of  U-2  and  RQ-4  aircraft,  in 
fact,  high  numbers  of  all  aircraft  (56,  68,  84,  and  84  total  aircraft,  respectively  out  of  a 
possible  84).  These  design  points,  while  possibly  unrealistic  in  terms  of  available  ISR 
platforms,  do  not  violate  any  of  the  assumptions  of  the  modeling  scenario  and  do  not 
warrant  removal  from  the  data  set. 
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3 
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5 

4 

3 

2 
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5 
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2 

2 
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3 

6 

6 

3 

3 
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6 

6 

3 
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3 
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3 

3 
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3 
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6 

6 

3 

3 
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6 

6 

3 

3 

3 
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6 

3 

3 

3 

6 
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3 
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Table  15.  Penetrating  Scenario:  High  Outlying  Data  Points 


After  removing  the  single  base  outliers,  the  distribution  and  summary 
statistics  for  the  penetrating  scenario  data  were  recalculated  (See  Figure  64).  The  mean 
coverage  increases  by  approximately  1%  and  the  remaining  data  more  closely 
approximates  a  normal  distribution. 


Coverage 


Quantiles 

Moments 

100.0% 

maximum 

0.78934 

Mean 

0.5929633 

99.5% 

0.77708 

Std  Dev 

0.0556522 

97.5% 

0.71091 

Std  Err  Mean 

0.0033869 

90.0% 

0.66040 

upper  95%  Mean 

0.5996315 

75.0% 

quartile 

0.62300 

lower  95%  Mean 

0.5862951 

50.0% 

median 

0.59176 

N 

270 

25.0% 

quartile 

0.56045 

10.0% 

0.52025 

2.5% 

0.48639 

0.5% 

0.41733 

0.0% 

minimum 

0.40478 

Figure  64.  Penetrating  Scenario:  Distribution  and  Summary  Statistics  of  Mission 
Area  Coverage  ( single  base  outliers  removed) 
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From  the  final  270  design  point  dataset  a  full  quadratic  model  with  main 
effect,  interaction,  and  polynomial  terms  was  created.  The  Stepwise  Regression  Control 
settings  within  JMP  were  as  follows: 

•  Probability  to  enter:  0.01 

•  Probability  to  leave:  0.01 

•  Direction:  Forward 

The  Construct  Model  Effects  macros  for  Factorial  to  Degree  and  Polynomial  to  Degree 
were  set  at  2  to  allow  the  possibility  of  two-way  interaction  and  quadratic  terms. 

The  resultant  model  achieves  an  R-squared  of  0.71  and  contains  16  main 
effect  terms,  7  interaction  terms,  and  4  second  order  terms.  See  Figures  65  and  66  for  the 
regression  plot  and  a  list  of  the  regression  model  terms.  Note  that  the  optimization 
interval  (OPT  INT)  is  clearly  the  most  significant  factor  in  the  model. 


Figure  65.  Actual  by  Predicted  Coverage  Plot  for  Full  Quadratic  Model 
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Parameter  Estimates 

Term 

Estimate 

Std  Error 

t  Ratio 

Prot»|t| 

Intercept 

0.3606258 

0.011655 

30.94 

<.0001* 

A  RQ-4 

0.0054156 

0.001988 

2.72 

0.0069* 

A  RQ-1 

0.0065338 

0.001058 

6.17 

<.0001* 

A  P-3 

0.0061525 

0.001063 

5.79 

<.0001* 

A  U-2 

0.0059377 

0.001998 

2.97 

0.0033* 

B  RQ-4 

0.0080442 

0.00197 

4.08 

<.0001* 

B  P-3 

0.003962 

0.001046 

3.79 

0.0002* 

B  U-2 

0.0064787 

0.001974 

3.28 

0.0012* 

C  RQ-4 

0.0107559 

0.001982 

5.43 

<.0001* 

C  P-3 

0.0022811 

0.001052 

2.17 

0.0311* 

C  U-2 

0.0050889 

0.001977 

2.57 

0.0107* 

D  RQ-4 

0.0063526 

0.001993 

3.19 

0.0016* 

D  RQ-1 

0.003488 

0.001069 

3.26 

0.0013* 

D  P-3 

0.0081541 

0.001063 

7.67 

<.0001* 

D  RC-135 

0.0071914 

0.002009 

3.58 

0.0004* 

D  U-2 

0.0136722 

0.002008 

6.81 

<.0001* 

OPT  INT 

0.0139716 

0.001338 

10.45 

<.0001* 

(A  RQ-4-1 ,52963)*(OPT  INT-3.46986) 

-0.003346 

0.001438 

-2.33 

0.0208* 

(A  RQ-1-3.05926)*(D  U-2-1. 52963) 

0.0032962 

0.001103 

2.99 

0.0031* 

(A  P-3-3. 05926)*(C  RQ-4-1 .52963) 

0.0009105 

0.000949 

0.96 

0.3383 

(A  U-2-1. 52963)*(D  RQ-4-1 .52963) 

-0.004324 

0.001949 

-2.22 

0.0274* 

(B  RQ-4-1 ,52963)*(B  U-2-1 .52963) 

-0.003686 

0.002221 

-1.66 

0.0983 

(C  RQ-4-1. 52963)*(OPT  INT-3.46986) 

-0.003959 

0.001359 

-2.91 

0.0039* 

(C  P-3-3. 05926)*(C  U-2-1. 52963) 

-0.001152 

0.001217 

-0.95 

0.3448 

(D  RQ-1  -3.05926)*(D  RC-1 35-1 .52963) 

-0.003878 

0.001108 

-3.50 

0.0005* 

(D  P-3-3. 05926)*(OPT  INT-3.46986) 

0.0006726 

0.000747 

0.90 

0.3688 

(B  P-3-3. 05926)*(B  P-3-3.05926) 

-0.001511 

0.000673 

-2.25 

0.0256* 

(D  U-2-1. 52963)*(D  U-2-1. 52963) 

-0.008468 

0.00225 

-3.76 

0.0002* 

Figure  66.  Full  Quadratic  Coverage  Model  Terms 


The  full  quadratic  model  for  the  penetrating  scenario  differs  significantly 
from  the  non-penetrating  scenario  model  in  the  number  of  terms  included.  In  the 
penetrating  model  all  of  the  platform  types  show  up  at  least  once,  where  as  in  the  non¬ 
penetrating  model,  P-3  were  not  represented  at  all.  The  main  similarity  between  the  non¬ 
penetrating  model  and  the  penetrating  model  is  that  the  U-2  and  RQ-4  are  represented  at 
every  base  as  significant  dependent  variables  in  the  regression  equation. 

To  test  the  validity  of  the  Coverage  Full  Quadratic  Regression  model  the 
regression  assumptions  from  Part  A  of  this  chapter  must  be  verified.  The  Actual  by 
Predicted  Plot  in  Figure  65  indicates  that  the  relationship  between  the  response  variable 
(coverage)  and  the  regressors  is  nearly  linear.  The  apparently  random  cloud  of  data 
points  with  a  mean  of  zero  in  the  Residual  by  Predicted  Plot,  Figure  67,  confirms  that  the 
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requirements  for  a  zero  mean,  constant  variance,  and  lack  of  correlation  of  the  residuals  is 
met.  Compliance  with  the  normality  assumption  is  shown  in  Figure  68  by  a  histogram  of 
the  residuals  overlaid  with  a  normal  curve  and  a  normal  quantile  plot. 


Residual  by  Predicted  Plot 


Figure  67.  Residual  by  Predicted  Plot  for  Coverage  Model 


Figure  68.  Histogram  and  Normal  Quantile  Plot  of  Residuals  for  the  Coverage  Model 
b.  Coverage  Factor  Interactions 

The  Interaction  Profile  matrix  for  the  coverage  model  is  16  x  16  cells,  too 
large  to  meaningfully  display  in  this  context.  The  matrix  contains  18  cells  with 
significant  interactions,  however,  just  as  in  the  non-penetrating  scenario,  the  minimum 
and  maximum  factor  setting  levels  result  in  only  marginal  changes  to  overall  coverage. 
Given  the  minimal  effect  of  the  interactions,  additional  detailed  discussion  of  the 
individual  interactions  is  unwarranted  for  this  study. 
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c. 


Coverage  Regression  Tree  Analysis 


Just  as  with  the  non-penetrating  scenario,  to  determine  the  number  of 
splits  that  would  continue  to  yield  additional  explanatory  value  for  Coverage,  20  splits 
were  done  in  rapid  succession  to  record  the  R-squared  value  without  regard  to  the  content 
or  structure  of  the  regression  tree.  These  R-squared  values  were  plotted  to  find  the  point 
that  the  curve  began  to  plateau,  indicating  diminishing  returns  from  each  successive  split. 
In  this  case,  10  splits  are  warranted.  Figure  69  provides  a  plot  of  the  R-squared  values 
against  the  number  of  splits. 


Penetrating  CART  Analysis 
R-Squared  vs.  Number  of  Splits 


Number  of  Regression  Tree  Splits 

Figure  69.  R-Squared  vs.  Number  of  Regression  Tree  Splits 

Figure  70  shows  the  regression  tree  for  Coverage  with  10  splits.  The  full 
tree  and  partition  graph  can  be  found  in  Appendix  J.  The  first  split  indicates  that  better 
coverage  is  obtained  when  the  optimization  interval  is  2. 1  hours  or  greater.  In  the  second 
split,  coverage  is  improved  by  having  one  or  more  U-2s  at  Base  D.  The  third  partition 
demonstrates  improved  coverage  when  there  are  three,  the  maximum  number  of  RQ-4s, 
at  Base  C.  The  fourth  split  indicates  that  better  coverage  is  gained  by  having  at  least  one 
RQ-4  at  Base  D.  The  fifth  split  indicates  that  better  coverage  is  obtained  with  the 

maximum  number  (six)  P-3s  at  Base  A.  The  sixth  split  indicates  that  coverage  is 
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improved  by  the  stationing  at  least  five  RQ-ls  at  Base  C.  The  seventh  split  is  a  return  to 
the  optimization  interval,  showing  that  coverage  improves  with  optimization  intervals  of 
5.37  hours  or  more.  Note  that  six  hours  is  the  maximum  optimization  interval  for  this 
study.  The  eighth  split  is  on  having  two  or  more  RQ-ls  at  Base  D.  Split  nine  indicates 
improved  coverage  by  stationing  two  or  more  RC-135s  at  Base  D.  The  final  split  is  on 
RQ-ls  at  Base  A,  this  time  indicating  improved  coverage  with  six  platforms,  the 
maximum  number. 

As  in  the  penetrating  scenario,  optimization  interval  appears  twice  and  is 
the  most  significant  factor.  Interestingly,  the  optimization  interval  is  the  most  significant 
factor  in  both  the  penetrating  and  non-penetrating  scenarios;  however,  its  level  has  the 
opposite  effect  on  coverage.  In  the  non-penetrating  scenario,  as  the  optimization  interval 
increases,  coverage  goes  down.  The  opposite  occurs  in  the  penetrating  scenario,  where 
coverage  improves  as  the  optimization  interval  goes  up.  This  disparate  behavior  is  likely 
due  to  addition  of  mission  areas,  and  consequently,  more  reachable  missions  in  the 
penetrating  scenario.  A  greater  variety  of  mission  area  to  mission  assignment  options 
reduces  the  chance  of  the  most  capable  platforms  going  unused  due  to  assignment 
saturation  caused  by  shorter  optimization  intervals. 

In  this  regression  tree,  all  types  of  aircraft  are  represented;  however,  Base 
B  does  not  appear.  That  is  not  to  suggest  that  Base  B  is  irrelevant,  only  that  it  is  not  a 
contributor  to  the  most  significant  factors.  Recall  that  Base  B  is  furthest  from  the  COI. 
The  presence  of  all  platform  types  is  in  contrast  to  the  non-penetrating  scenario  where  U- 
2s  were  heavily  represented  and  P-3s  not  at  all. 
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Figure  70.  Regression  Tree  for  Coverage 


d.  Coverage  Contour  Plot  Analysis 

To  explore  the  relationship  between  the  various  types  of  ISR  platforms 
and  the  total  number  of  platforms,  contour  plots  were  generated.  The  various  color  filled 
regions  of  the  plot  represent  the  amount  of  coverage  attained  as  the  number  of  each 
platform  and  total  aircraft  vary.  Recall  from  the  non-penetrating  scenario  contour  plot 
analysis  that  the  combinatorial  nature  of  the  problem  and  strong  interactions  of  the 
regressors  result  in  a  non-smooth  contour  plot.  Also,  note  that  the  surface  is  affected  by 
changes  in  other  factors;  therefore,  trends  in  the  contour  plots  are  more  important  than 
specific  values. 

Figure  7 1  shows  that  in  order  achieve  the  best  coverage,  approximately  60 
total  aircraft  are  required  with  at  least  4  U-2s.  Once  60  total  aircraft  are  available,  the 
addition  of  more  U-2  platforms  continues  to  improve  coverage. 
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Contour  Plot  for  Coverage 


—  Legend 
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Figure  7 1 .  Coverage  Contour  Plot  for  U-2  vs.  Total  Aircraft 


The  relationship  between  the  number  of  RQ-4s  and  total  aircraft  is 
illustrated  in  Figure  72.  The  total  number  of  aircraft  necessary  to  achieve  the  best  levels 
of  coverage  is  lower  than  for  the  U-2  once  there  are  four  or  more  RQ-4s  in  the 
simulation.  As  with  the  U-2,  coverage  continues  to  improve  with  the  addition  of  more 
RQ-4  airframes. 


Contour  Plot  for  Coverage 


Legend 
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Figure  72.  Coverage  Contour  Plot  for  RQ-4  vs.  Total  Aircraft 


Figure  73  demonstrates  the  value  of  the  RC-135  in  relation  to  total 
aircraft.  The  maximum  levels  of  coverage  appear  to  be  achieved  when  there  are  at  least 
55  total  aircraft  and  3  or  more  RC-135s.  The  RC-135  continues  to  maximize  coverage  as 
its  quantity  and  the  total  number  of  aircraft  rise. 
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Contour  Plot  for  Coverage 


—  Legend 
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Figure  73.  Coverage  Contour  Plot  for  RC-135  vs.  Total  Aircraft 


Figure  74  displays  the  relationship  between  the  quantity  of  P-3  airframes 
and  the  total  number  of  aircraft.  The  best  levels  of  coverage  are  reached  when  there  are 
more  than  10  P-3s  and  60  or  more  total  aircraft.  Beyond  these  levels,  coverage  continues 
to  improve. 

Contour  Plot  for  Coverage 
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Figure  74.  Coverage  Contour  Plot  for  P-3  vs.  Total  Aircraft 


Figure  75  shows  that  the  best  coverage  is  attained  with  approximately  60 
total  aircraft  and  more  than  12  RQ-ls.  As  with  all  of  the  other  platform  types,  coverage 
continues  to  rise  with  the  addition  of  more  airframes. 
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Contour  Plot  for  Coverage 


—  Legend 
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Figure  75.  Coverage  Contour  Plot  for  RQ-1  vs.  Total  Aircraft 

4.  Coverage  by  Type 


The  Coverage  by  Type  MOE  is  a  subset  of  total  coverage.  Figure  76  displays  the 
amount  of  coverage  by  each  platform  type.  Recall  that  mean  coverage  for  the  non¬ 
penetrating  scenario  was  59.2%.  Of  this  total  amount  of  coverage  the  RQ-4  is  the  largest 
contributor  at  52.3%  of  the  total,  far  exceeding  the  next  closest  contributor,  the  U-2  at 
19.5%.  In  the  penetrating  scenario,  these  two  platforms  account  for  nearly  72%  of  the 
total  coverage.  Again,  this  is  not  unexpected  given  the  superior  sensor  packages  carried 
by  these  airframes.  The  ability  of  the  RQ-4  to  remain  on  station,  along  with  its  highly 
capable  sensor  package,  clearly  demonstrates  its  value.  The  ability  to  use  mission  areas 
in  country  appears  to  have  boosted  the  P-3s  contribution  in  the  penetrating  scenario.  The 
utility  of  the  RQ-1  declines  slightly  in  the  penetrating  scenario,  likely  due  to  the 
enhanced  collection  ability  of  the  other  platforms  given  the  addition  of  the  penetrating 
mission  areas.  The  RC-135s  total  contribution  appears  severely  limited  by  its  single 
sensor  capability.  Also,  as  in  the  non-penetrating  scenario,  this  disadvantage  is 
compounded  by  the  fact  that  there  are  fewer  SIGINT  targets  in  the  scenario  than  EO/IR 
or  SAR  targets. 
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Figure  76.  Coverage  by  Platform  Type 

5.  Attrition 

Attrition  is  a  relatively  uninteresting  MOE  for  this  analysis.  The  lack  of  realistic 
unclassified  performance  data  and  parameters  for  the  threat  platforms  and  vulnerability 
data  for  the  ISR  platforms  resulted  in  a  generic  treatment  of  this  data  within  the 
simulation.  The  Pk  for  all  ISR  platforms  at  the  minimum  and  maximum  surface-to-air 
missile  range  was  set  at  0.6  and  0.5,  respectively.  Consequently,  the  ISR  platforms  were 
attrited  in  exact  proportion  to  their  presence  in  the  scenario  (See  Figure  77).  Of  note, 
4,139,948  aircraft  sorties  were  generated  and  flown  in  this  simulation,  2%  or  94,390  were 
killed.  Attrition  for  studies  conducted  in  a  classified  environment  with  real  world 
parameters  would  likely  differ  significantly  from  the  results  found  in  this  experiment. 
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Attrition  Proportion  of  ISR  Platforms 


Figure  77.  Proportional  Attrition  of  ISR  Platforms 

6.  Comparison  of  Non-Penetrating  vs.  Penetrating  Results 

Several  significant  similarities  and  differences  between  the  output  from  the  non¬ 
penetrating  and  penetrating  scenarios  were  identified.  These  findings  and  an  explanation 
of  the  underlying  factors  is  provided. 

a.  Coverage 

As  expected,  the  coverage  for  the  penetrating  scenario  exceeds  that  of  the 
non-penetrating  scenario  (See  Figure  78).  The  non-penetrating  scenario  resulted  in  a 
mean  coverage  of  0.49,  with  a  standard  deviation  of  0.02,  and  maximum  and  minimum 
values  of  0.53  and  0.43,  respectively.  The  penetrating  scenario  yielded  a  mean  coverage 
of  0.59,  with  a  standard  deviation  of  0.06,  and  maximum  and  minimum  values  of  0.79 
and  0.40,  respectively.  The  reduced  coverage  in  the  non-penetrating  scenario  is  due  to 
the  physical  inability  of  the  sensors  to  range  the  targets.  It  must  be  noted  that  coverage 
for  the  penetrating  scenario  could  potentially  be  much  lower  if  the  attrition  rate  was 
increased. 

While  the  penetrating  scenario  provides  10%  more  mean  coverage,  its 
results  contain  3  times  the  variability.  In  fact,  the  lowest  coverage  value  is  actually  found 
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in  the  results  of  the  penetrating  scenario.  This  variability  is  due  to  both  the  stochastic 
nature  of  the  penetrating  scenario  and  relative  increased  complexity  of  the  approximate 
dynamic  programming  approach  due  to  the  addition  of  more  mission  areas. 


Non-Penetrating  Coverage  Penetrating  Coverage 


Figure  78.  Penetrating  vs.  Non-Penetrating  Coverage 
c.  Coverage  by  Type 

A  comparison  of  the  coverage  by  platform  type  for  each  scenario  is  shown 
in  Figure  79.  The  RQ-4  provides  the  greatest  amount  of  coverage  in  both  scenarios; 
however  it  is  nearly  matched  by  the  U-2  in  the  non-penetrating  scenario.  This  disparity  is 
likely  due  to  the  RQ-4s  ability  to  remain  on-station,  particularly  in  the  penetrating 
scenario  where  additional  transit  time  is  required  to  reach  the  mission  areas.  The  RQ-4 
and  U-2  superior  coverage  is  not  unexpected  as  both  airframes  carry  all  three  of  the 
sensor  types  considered  for  this  study.  The  substantial  demand  for  the  three  sensor  types 
impacts  the  value  of  the  most  capable  platforms.  The  combined  coverage  of  the 
remaining  platforms,  the  RQ-1,  the  P-3,  and  the  RC-135,  is  approximately  25%  of  the 
total  in  both  scenarios.  None  of  these  aircraft  has  more  than  two  sensor  types,  lowering 
their  ability  to  contribute  significantly  to  coverage.  Of  note,  the  P-3  provides  almost  no 
value  in  the  non-penetrating  scenario,  likely  due  to  its  relatively  short  on  station  time 
when  compared  to  the  RQ-1  which  has  a  similar  sensor  package. 
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Figure  79.  Non-Penetrating  vs.  Penetrating  Coverage  by  Platform  Type 
b.  Factor  Comparison 

Figure  80  illustrates  the  opposite  effect  demonstrated  by  the  optimization 
interval  factor  with  respect  to  coverage.  In  the  non-penetrating  scenario,  shorter 
optimization  intervals  are  better,  but  in  the  penetrating  scenario  longer  optimization 
intervals  yield  improved  coverage.  This  disparity  is  explained  by  the  increase  in  mission 
areas  and  reachable  missions  available  in  the  penetrating  scenario.  With  more  mission 
areas  and  missions  included  in  each  optimization,  there  is  less  chance  that  the  more 
capable  airframes  will  be  underutilized  due  to  assignment  saturation.  Recall  from 
Chapter  II  that  all  open  missions  and  available  airframes  are  matched  at  each 
optimization  event.  When  there  are  a  greater  number  of  mission  areas  and  missions 
available,  as  in  the  penetrating  scenario,  there  is  less  chance  that  the  better  platforms  will 
be  assigned  to  low  target  density  areas  or  not  used  at  all. 
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Bivariate  Fit  of  Coverage  By  OPT  INT 


Non-Penetrating  Scenario 


Bivariate  Fit  of  Coverage  By  OPT  INT 


Penetrating  Scenario 


Figure  80.  Bivariate  Fit  of  Coverage  by  Optimization  Interval  for  the  Non- 
Penetrating  and  Penetrating  Scenarios 


The  U-2  and  the  RQ-4  show  up  as  significant  factors  in  both  scenarios. 
An  examination  of  the  regression  trees  from  each  scenario,  Figures  60  (page  90)  and  70 
(page  99),  reveals  that  the  dataset  is  partitioned  on  U-2  as  a  factor  before  the  RQ-4  even 
though  the  RQ-4  provides  a  greater  percentage  of  the  overall  coverage  in  both  scenario. 
This  is  due  to  the  RQ-4s  long  endurance.  The  long  dwell  time  of  the  RQ-4  means  that  it 
is  involved  in  fewer  optimization  events  and  therefore  does  not  contribute  to  the 
variability  of  the  coverage  in  the  same  way  as  the  U-2. 
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VI.  FINDINGS  AND  RECOMMENDATIONS 


A.  SIGNIFICANT  FINDINGS 

JDAFS  was  exercised  to  determine  its  viability  for  use  in  future  studies.  In  doing 
so,  scenarios  were  constructed  and  simulations  run  based  on  Nearly  Orthogonal  Latin 
Hypercube  design  of  experiment  methodology.  Based  on  these  experiments,  data 
analysis  was  conducted  on  the  resultant  simulation  output  and  issues  with  the  JDAFS 
model  were  identified  for  enhancement  or  correction.  The  significant  findings  from  this 
research  are  summarized  below. 

1.  Data  Analysis  Issues 

•  Optimization  is  the  most  significant  factor  in  both  the  scenarios,  but  has 
the  opposite  effect.  In  the  non-penetrating  scenario,  shorter  optimization 
intervals  are  better,  but  in  the  penetrating  scenario  longer  optimization 
intervals  yield  improved  coverage.  Increased  assignment  options  in  the 
penetrating  scenario  causes  this  disparate  behavior  by  the  optimization 
interval  factor. 

•  The  U-2  is  the  most  important  platform  factor  in  explaining  the  variability 
in  both  scenarios. 

•  The  RQ-4  results  in  the  majority  of  mission  coverage  for  both  scenarios 
due  to  its  ability  to  remain  on  station. 

•  Coverage  is  not  necessarily  improved  by  adding  more  ISR  assets  in  the 
non-penetrating  scenario. 

•  Coverage  is  improved  by  the  addition  of  assets  in  the  penetrating  scenario. 
Coverage  would  likely  eventually  also  plateau  with  diminishing  returns 
from  the  addition  of  more  aircraft  in  this  scenario.  The  maximum  number 
of  airframes  in  the  scenario  was  not  sufficient  to  explore  this  behavior. 

•  Penetrating  ISR  platforms  average  20%  more  mean  coverage  than  non¬ 
penetrating  platforms. 

2.  Issues  Identified  and  Corrected  During  Preliminary  Testing 

The  exercise  of  the  JDAFS  model  prior  to  the  commencement  of  production  runs 
identified  a  number  of  bugs  and  deficiencies.  These  discrepancies  were  all  promptly 
addressed  and  corrected  by  the  model  developers.  The  following  is  a  summary  of  the 
issues  and  modifications  made  to  JDAFS. 


Ill 


a. 


JDAFS  Architectural  Issues 


Initial  attempts  to  run  JDAFS  in  a  cluster  environment  were  unsuccessful. 
JDAFS  was  originally  written  to  be  run  from  a  GUI.  The  basic  window  object  for  a  GUI 
in  the  Java  programming  language  is  called  a  "JFrame".  The  model  initialization  code  for 
JDAFS  was  placed  inside  a  JFrame  constructor.  Thus,  even  if  no  window  was  opened,  a 
JFrame  needed  to  be  constructed  to  run  JDAFS.  In  order  to  construct  a  JFrame,  Java 
polls  the  environment  to  determine  all  of  the  characteristics  of  the  GUI  enviromnent. 
Since  there  wasn’t  any  such  environment  when  running  JDAFS  from  the  command  line  in 
a  cluster  environment,  this  caused  a  "Null  pointer  exception",  i.e.,  a  reference  to 
something  which  doesn’t  exist.  This  is  a  tenninal  error  for  a  Java  program.  The  model 
developers  removed  the  model  initialization  code  from  JDAFS  and  built  a  completely 
new  installer  that  ensured  cluster  compatibility. 

Aircraft  routing  was  initially  done  on  a  straight  line  path  from  a  base  to  its 
associated  mission  areas.  To  prevent  the  overflight  of  denied  areas,  it  was  necessary  to 
implement  a  waypoint  routing  capability.  The  “route”  table  was  added  to  give  the  ability 
to  specify  ingress  and  egress  routes  between  bases  and  mission  areas  or  between  two 
mission  areas. 


b.  JDAFS  Bugs  Identified  and  Corrected 

JDAFS  uses  the  Bugzilla  open-source  defect  tracking  system  to  allow 
users  to  report  issues  discovered  when  using  the  simulation.  Bugzilla  also  provides  a 
means  for  the  JDAFS  developers  to  communicate  the  status  of  correcting  any 
deficiencies.  All  of  the  bugs  identified  in  this  research  were  promptly  addressed  and 
corrected,  often  in  less  than  24  hours.  Table  16  list  the  bugs  identified,  their  symptoms, 
and  the  necessary  corrections  made  in  JDAFS. 
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Symptom 

Problem 

Correction 

JDAFS  simulation  failed  to 
run  to  completion.  An 

Illegal  State  Transition 
Exception  error  was  thrown 
in  JAVA. 

If  a  mission  platfonn  was 
killed  on  the  ground,  its 
pending  events  were  not 
cancelled.  Since  a  dead 
platform  is  unable  to 
execute  its  mission,  JAVA 
throws  the  Illegal  State 
Transition  Exception  and 
the  simulation  cannot 
continue. 

The  JDAFS  JAVA  code 
was  amended  to  ensure  all 
pending  events  are 
cancelled  for  killed  mission 
platforms. 

JDAFS  Simulation 
optimization  event  took 
successively  longer  to 
complete,  eventually 
resulting  in  a  failure  to  run 
to  completion. 

JDAFS  was  not  calling 
deleteLp()  following 
optimization  events.  This 
resulted  in  a  memory  leak 
in  lpsolve.  As  the 
simulation  progressed,  there 
were  insufficient  memory 
resources  for  the  program  to 
continue. 

A  call  to  deleteLp()  was 
added  in  the  3D  CVO 
process  in  JAVA  to  ensure 
that  memory  resources  were 
freed  prior  to  each 
optimization  event. 

Unusually  high  coverage 
values  were  reported 
following  simulation  runs. 

Platforms  were  receiving 
credit  for  coverage  of 
targets  that  were  out  of 
range  or  for  which  LOS  did 
not  exist. 

JDAFS  coverage  reports 
were  corrected  to  ensure 
coverage  credit  was  only 
received  when  platforms 
actually  covered  a  mission. 

Table  16.  JDAFS  Bugs  Identified  and  Corrected 


B.  RECOMMENDATIONS  FOR  FURTHER  JDAFS  IMPROVEMENT 

JDAFS  is  a  powerful  research  tool;  however,  a  number  of  modifications  are 
warranted  that  will  improve  the  usability,  functionality,  and  credibility  of  the  model  for 
future  studies.  The  following  changes  and  improvements  are  recommended: 

•  Allow  ISR  platfonns  to  receive  credit  for  collection  on  targets  within 
sensor  range  during  ingress  and  egress  to  mission  areas. 

•  Create  probability  distributions  to  represent  current  and  future  force 
sensors  from  certified  data  to  accurately  represent  the  acquisition  of 
targets  by  sensors  in  the  model. 

•  Implement  revisit  valuation  schemes  for  missions  to  more  credibly 
represent  ISR  mission  characteristics. 
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•  Addition  of  joint  assets  (platforms,  sensors,  munitions,  etc.)  in  the  JDAFS 
database  for  reuse  in  joint  studies. 

•  Further  develop  the  functionality  of  JDAFS  to  more  credibly  represent 
Joint  Sensors  to  include  satellite  assets. 

•  Provide  a  more  robust  representation  of  unmanned  aerial  vehicles  with 
weapons  and  sensors  that  allow  force-on-force  analysis  of  differing 
platform  mixes. 

•  Develop  an  integrated  design  of  experiments  interface  that  enables  a  quick 
determination  of  alternatives. 

•  Incorporate  the  InputGenerator  and  OutputGenerator  scripts  into  the 
JDAFS  program.  The  integration  of  these  tools  will  eliminate  the  need  for 
the  pre  and  post-processing  of  data  and  eliminate  the  need  to  transfer 
excessively  large  files. 

•  Develop  a  simple,  well  documented  JDAFS  interface  to  allow  an  analyst 
to  easily  enter  the  data  for  the  various  input  factors. 

•  Develop  a  comprehensive  JDAFS  Users/ Analyst  Manual. 

C.  RECOMMENDATIONS  FOR  FUTURE  JDAFS  STUDIES 

The  research  conducted  for  this  thesis  exercised  the  JDAFS  model  and 
demonstrated  its  utility  as  a  simulation  and  data  farming  tool.  The  following  areas  should 
be  considered  for  additional  research  using  JDAFS: 

•  Exercise  data  farming  concepts  as  additional  development  is  done  to  the 
JDAFS  model. 

•  Explore  the  use  of  JDAFS  to  study  UAV  acquisition  of  moving  or  pop-up 
targets. 

•  Conduct  analysis  using  real  world  terrain  and  order  of  battle  to  explore 
multiple  operational  factors. 

•  Explore  the  use  of  JDAFS  as  an  acquisition  and  capabilities  requirements 
tool. 

•  Use  JDAFS  to  explore  the  trade-offs  associated  with  manned  vs. 
unmanned  ISR  platforms. 

•  Evaluate  the  utility  of  JDAFS  as  a  screening  tool  for  high  resolution 
modeling  and  simulation. 

•  Allow  cueing  and  flexible  time  windows  to  further  enhance  scenario 
realism. 
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APPENDIX  A.  NON-PENETRATING  SCENARIO  BASE  TO 
MISSION  AREA  DISTANCE  TABLE 


BASE 

MISSION 

AREA 

A 

B 

C 

D 

1 

328000 

1195861 

1250502 

1620154 

2 

356629 

1061925 

1110502 

1480154 

3 

424834 

939140 

980502 

1350154 

4 

569745 

936862 

861140 

1205242 

5 

714656 

956785 

750824 

1060331 

6 

859568 

997578 

654149 

915420 

7 

1004479 

1056827 

578000 

770509 

8 

1149390 

1199525 

654149 

625629 

9 

1294301 

1342698 

750824 

480767 

10 

1439212 

1486207 

861140 

335948 

11 

1584124 

1629964 

980502 

191269 

12 

1714124 

1759964 

1 092742 

187840 

13 

1854124 

1899964 

1217573 

267739 

24 

356629 

1331046 

1390502 

1760154 

MIN 

328000 

936862 

578000 

187840 

MAX 

1854124 

1899964 

1390502 

1760154 

MEAN 

974987 

1271025 

938075 

875094 

MEDIAN 

932023 

1197693 

920821 

842965 

Non-Penetrating  Scenario:  Base  to  Mission  Area  Distance  ( in  meters)  Table  ( The 
additional  transit  required  due  to  waypoint  routing  to  prevent  COI  overflight  is  included 
in  the  distance  calculations  for  this  table). 
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APPENDIX  B.  PENETRATING  SCENARIO  BASE  TO  MISSION 

AREA  DISTANCE  TABLE 


BASE 

MISSION 

AREA 

A 

B 

C 

D 

1 

328000 

1195861 

1217573 

1238258 

2 

356629 

1061925 

1 092742 

1223472 

3 

424834 

939140 

980502 

1224003 

4 

566667 

936862 

861140 

1099573 

5 

709744 

956785 

750824 

980784 

6 

853441 

997578 

654149 

869949 

7 

997489 

1056827 

578000 

770509 

8 

1082618 

1199525 

654149 

625629 

9 

1179465 

1342698 

750824 

480767 

10 

1285384 

1486207 

861140 

335948 

11 

1398315 

1629964 

980502 

191269 

12 

1379124 

1703668 

1 092742 

187840 

13 

1372000 

1790219 

1217573 

267739 

24 

356629 

1331046 

1345394 

1268418 

51 

450000 

1234909 

1170470 

1118034 

52 

672301 

1 504024 

1329018 

1000744 

53 

672301 

1134411 

912956 

912079 

54 

700000 

1346291 

1110180 

873212 

55 

939415 

1767767 

1500000 

921954 

56 

863134 

1553222 

1250000 

782624 

57 

863134 

1312440 

950000 

701783 

58 

939415 

1133578 

700000 

728011 

59 

1000000 

1523975 

1110180 

585235 

60 

1082953 

1727104 

1329018 

638661 

61 

1082953 

1416929 

912956 

488148 

62 

1250000 

1698529 

1170470 

360555 

MIN 

328000 

936862 

578000 

187840 

MAX 

1398315 

1790219 

1500000 

1268418 

MEAN 

877152 

1 345442 

1018558 

764431 

MEDIAN 

901274 

1336872 

1036622 

776566 

Penetrating  Scenario:  Base  to  Mission  Area  Distance  (in  meters)  Table 
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APPENDIX  C.  BASE  ‘A’  TRANSIT  AND  ON-STATION 

STATISTICS 


NON-PENETRATING  SCENARIO 


MISSION 

RQ-4 

RQ-1 

RC-135 

U-2 

AREA 

TRANS. 

ON-STA 

TRANS. 

ON-STA 

TRANS. 

ON-STA 

TRANS. 

ON-STA 

1 

1.041799 

34.9582 

4.427646 

19.57235 

1 .55356 

10.44644 

0.814271 

10.18573 

0.795981 

9.204019 

2 

1.13273 

34.86727 

4.814102 

19.1859 

1.689159 

10.31084 

0.885342 

10.11466 

0.865456 

9.134544 

3 

1 .349365 

34.65063 

5.734801 

18.2652 

2.012211 

9.987789 

1 .054664 

9.945336 

1.030976 

8.969024 

4 

1.809634 

34.19037 

7.690946 

16.30905 

2.698577 

9.301423 

1.414411 

9.585589 

1 .382642 

8.617358 

5 

2.269904 

33.7301 

9.64709 

14.35291 

3.384944 

8.615056 

1.774158 

9.225842 

1.734308 

8.265692 

6 

2.730173 

33.26983 

11.60323 

12.39677 

4.07131 

7.92869 

2.133904 

8.866096 

2.085975 

7.914025 

7 

3.190442 

32.80956 

13.55938 

10.44062 

4.757677 

7.242323 

2.493651 

8.506349 

2.437641 

7.562359 

8 

3.650711 

32.34929 

15.51552 

8.484476 

5.444043 

6.555957 

2.853398 

8.146602 

2.789308 

7.210692 

9 

4.110981 

31.88902 

17.47167 

6.528332 

6.13041 

5.86959 

3.213144 

7.786856 

3.140974 

6.859026 

10 

4.57125 

31.42875 

19.42781 

4.572188 

6.816776 

5.183224 

3.572891 

7.427109 

3.49264 

6.50736 

11 

5.031519 

30.96848 

21.38396 

2.616043 

7.503143 

4.496857 

3.932638 

7.067362 

3.844307 

6.155693 

12 

5.444427 

30.55557 

23.13882 

0.861184 

8.118883 

3.881117 

4.255367 

6.744633 

4.159787 

5.840213 

13 

5.889098 

30.1109 

24 

0 

8.781988 

3.218012 

4.602922 

6.397078 

4.499535 

5.500465 

24 

1.13273 

34.86727 

4.814102 

19.1859 

1.689159 

10.31084 

0.885342 

10.11466 

0.865456 

9.134544 

MIN 

30.11 

4.43 

0.00 

i 

MAX 

5.89 

34.96 

25.03 

19.57 

MEAN 

3.10 

32.90 

13.16 

10.84 

7.38 

Mai 

MEDIAN 

2.96 

33.04 

12.58 

11.42 

m&i 

PENETRATING  SCENARIO 

MISSION 

RQ-4 

RQ-1 

RC-135 

U-2 

AREA 

TRANS. 

ON-STA 

TRANS. 

ON-STA 

TRANS. 

ON-STA 

TRANS. 

ON-STA 

1 

1.041799 

34.9582 

4.427646 

19.57235 

1.55356 

10.44644 

0.814271 

10.18573 

0.795981 

9.204019 

2 

1.13273 

34.86727 

4.814102 

19.1859 

1.689159 

10.31084 

0.885342 

10.11466 

0.865456 

9.134544 

3 

1 .349365 

34.65063 

5.734801 

18.2652 

2.012211 

9.987789 

1 .054664 

9.945336 

1.030976 

8.969024 

4 

1.799856 

34.20014 

7.64939 

16.35061 

2.683996 

9.316004 

1 .406769 

9.593231 

1.375171 

8.624829 

5 

2.254302 

33.7457 

9.580782 

14.41922 

3.361678 

8.638322 

1.761963 

9.238037 

1 .722388 

8.277612 

6 

2.710713 

33.28929 

11.52053 

12.47947 

4.042291 

7.957709 

2.118694 

8.881306 

2.071106 

7.928894 

7 

3.168241 

32.83176 

13.46502 

10.53498 

4.724569 

7.275431 

2.476298 

8.523702 

2.420678 

7.579322 

8 

3.438628 

32.56137 

14.61417 

9.385829 

5.127779 

6.872221 

2.687634 

8.312366 

2.627267 

7.372733 

9 

3.746235 

32.25376 

15.9215 

8.0785 

5.586491 

6.413509 

2.928059 

8.071941 

2.862292 

7.137708 

10 

4.082657 

31.91734 

17.35129 

6.648708 

6.088173 

5.911827 

3.191007 

7.808993 

3.119333 

6.880667 

11 

4.44135 

31.55865 

18.87574 

5.124262 

6.623066 

5.376934 

3.471361 

7.528639 

3.393391 

6.606609 

12 

4.380398 

31.6196 

18.61669 

5.38331 

6.532172 

5.467828 

3.423721 

7.576279 

3.346821 

6.653179 

13 

4.357769 

31.64223 

18.52052 

5.479482 

6.498427 

5.501573 

3.406034 

7.593966 

3.329531 

6.670469 

1.13273 

34.86727 

4.814102 

19.1859 

1.689159 

10.31084 

0.885342 

10.11466 

0.865456 

9.134544 

1 .429297 

34.5707 

6.074514 

17.92549 

2.131408 

9.868592 

1.11714 

9.88286 

1 .092047 

8.907953 

52 

2.135372 

33.86463 

9.075331 

14.92467 

3.184327 

8.815673 

1 .669008 

9.330992 

1.63152 

8.36848 

53 

2.135372 

33.86463 

9.075331 

14.92467 

3.184327 

8.815673 

1 .669008 

9.330992 

1.63152 

8.36848 

54 

2.223352 

33.77665 

9.449244 

14.55076 

3.315524 

8.684476 

1 .737773 

9.262227 

1.698741 

8.301259 

55 

2.983784 

33.01622 

12.68108 

11.31892 

4.449503 

7.550497 

2.332127 

8.667873 

2.279745 

7.720255 

56 

2.7415 

33.2585 

11.65137 

12.34863 

4.088202 

7.911798 

2.142758 

8.857242 

2.094629 

7.905371 

57 

2.7415 

33.2585 

11.65137 

12.34863 

4.088202 

7.911798 

2.142758 

8.857242 

2.094629 

7.905371 

58 

2.983784 

33.01622 

12.68108 

11.31892 

4.449503 

7.550497 

2.332127 

8.667873 

2.279745 

7.720255 

59 

3.176216 

32.82378 

13.49892 

10.50108 

4.736463 

7.263537 

2.482532 

8.517468 

2.426772 

7.573228 

60 

3.439694 

32.56031 

14.6187 

9.381299 

5.129369 

6.870631 

2.688467 

8.311533 

2.628081 

7.371919 

61 

3.439694 

32.56031 

14.6187 

9.381299 

5.129369 

6.870631 

2.688467 

8.311533 

2.628081 

7.371919 

62 

3.970271 

32.02973 

16.87365 

7.12635 

5.920579 

6.079421 

3.103165 

7.896835 

3.033465 

6.966535 

MIN 

1.04 

31.56 

5.12 

1.55 

Hi 

7.53 

0.80 

6.61 

MAX 

4.44 

34.96 

mBM 

19.57 

6.62 

10.19 

3.39 

9.20 

MEAN 

2.79 

33.21 

M 

12.16 

4.15 

7.85 

8.82 

2.13 

7.87 

MEDIAN 

2.86 

33.14 

wmm 

11.83 

4.27 

7.73 

2.24 

8.76 

2.19 

7.81 
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APPENDIX  D.  BASE  ‘B’  TRANSIT  AND  ON-STATION 

STATISTICS 


NON-PENETRATING  SCENARIO 

B 

MISSION 

RQ-4 

RQ-1 

RC-135 

U-2 

AREA 

TRANS. 

ON-STA 

TRANS. 

ON-STA 

TRANS. 

ON-STA 

TRANS. 

ON-STA 

1 

3.798314 

32.20169 

16.14283 

7.857165 

5.664153 

6.335847 

2.968764 

8.031236 

2.902083 

7.097917 

2 

3.372903 

32.6271 

14.33484 

9.665164 

5.029767 

6.970233 

2.636262 

8.363738 

2.577049 

7.422951 

3 

2.982912 

33.01709 

12.67738 

1 1 .32262 

4.448202 

7.551798 

2.331445 

8.668555 

2.279079 

7.720921 

4 

2.975678 

33.02432 

12.64663 

1 1 .35337 

4.437415 

7.562585 

2.325791 

8.674209 

2.273552 

7.726448 

5 

3.038955 

32.96104 

12.91556 

1 1 .08444 

4.531776 

7.468224 

2.375249 

8.624751 

2.321899 

7.678101 

6 

3.168523 

32.83148 

13.46622 

10.53378 

4.72499 

7.27501 

2.476519 

8.523481 

2.420894 

7.579106 

7 

3.356712 

32.64329 

14.26603 

9.733972 

5.005624 

6.994376 

2.623608 

8.376392 

2.564679 

7.435321 

8 

3.809952 

32.19005 

16.1923 

7.807702 

5.681508 

6.318492 

2.977861 

8.022139 

2.910975 

7.089025 

9 

4.264698 

31.7353 

18.12497 

5.875032 

6.359638 

5.640362 

3.33329 

7.66671 

3.258421 

6.741579 

10 

4.720515 

31.27948 

20.06219 

3.93781 

7.039365 

4.960635 

3.689557 

7.310443 

3.606686 

6.393314 

11 

5.17712 

30.82288 

22.00276 

1.997241 

7.720266 

4.279734 

4.046439 

6.953561 

3.955552 

6.044448 

12 

5.590028 

30.40997 

23.75762 

0.242381 

8.336007 

3.663993 

4.369168 

6.630832 

4.271033 

5.728967 

13 

6.034698 

29.9653 

24 

0 

8.999112 

3.000888 

4.716723 

6.283277 

4.610781 

5.389219 

23 

4.227691 

31.77231 

17.96769 

6.032314 

6.304451 

5.695549 

3.304365 

7.695635 

3.230146 

6.769854 

MIN 

2.98 

29.97 

12.65 

0.00 

4.44 

3.00 

MAX 

6.03 

33.02 

25.65 

11.35 

9.00 

7.56 

wkM 

MEAN 

4.04 

31.96 

17.16 

6.84 

6.02 

5.98 

3.08 

MEDIAN 

3.80 

32.20 

16.17 

7.83 

5.67 

6.33 

2.91 

7.09 

PENETRATING  SCENARIO 

B 


MISSION 

RQ-4 

RQ-1 

RC-135 

U-2 

AREA 

TRANS. 

ON-STA 

TRANS. 

ON-STA 

TRANS. 

ON-STA 

TRANS. 

ON-STA 

1 

3.798314 

32.20169 

16.14283 

7.857165 

5.664153 

6.335847 

2.968764 

8.031236 

2.902083 

7.097917 

2 

3.372903 

32.6271 

14.33484 

9.665164 

5.029767 

6.970233 

2.636262 

8.363738 

2.577049 

7.422951 

3 

2.982912 

33.01709 

12.67738 

1 1 .32262 

4.448202 

7.551798 

2.331445 

8.668555 

2.279079 

7.720921 

4 

2.975678 

33.02432 

12.64663 

1 1 .35337 

4.437415 

7.562585 

2.325791 

8.674209 

2.273552 

7.726448 

5 

3.038955 

32.96104 

12.91556 

1 1 .08444 

4.531776 

7.468224 

2.375249 

8.624751 

2.321899 

7.678101 

6 

3.168523 

32.83148 

13.46622 

10.53378 

4.72499 

7.27501 

2.476519 

8.523481 

2.420894 

7.579106 

7 

3.356712 

32.64329 

14.26603 

9.733972 

5.005624 

6.994376 

2.623608 

8.376392 

2.564679 

7.435321 

8 

3.809952 

32.19005 

16.1923 

7.807702 

5.681508 

6.318492 

2.977861 

8.022139 

2.910975 

7.089025 

9 

4.264698 

31.7353 

18.12497 

5.875032 

6.359638 

5.640362 

3.33329 

7.66671 

3.258421 

6.741579 

10 

4.720515 

31.27948 

20.06219 

3.93781 

7.039365 

4.960635 

3.689557 

7.310443 

3.606686 

6.393314 

11 

5.17712 

30.82288 

22.00276 

1.997241 

7.720266 

4.279734 

4.046439 

6.953561 

3.955552 

6.044448 

12 

5.411218 

30.58878 

22.99768 

1 .002324 

8.06936 

3.93064 

4.22941 

6.77059 

4.134414 

5.865586 

13 

5.686123 

30.31388 

24 

0 

8.479306 

3.520694 

4.444276 

6.555724 

4.344454 

5.655546 

24 

4.227691 

31.77231 

17.96769 

6.032314 

6.304451 

5.695549 

3.304365 

7.695635 

3.230146 

6.769854 

51 

3.922338 

32.07766 

16.66994 

7.330063 

5.849101 

6.150899 

3.065701 

7.934299 

2.996843 

7.003157 

52 

4.777106 

31.22289 

20.3027 

3.697301 

7.123754 

4.876246 

3.733788 

7.266212 

3.649923 

6.350077 

53 

3.603134 

32.39687 

15.31332 

8.686678 

5.373095 

6.626905 

2.816212 

8.183788 

2.752957 

7.247043 

54 

4.276112 

31.72389 

18.17348 

5.826523 

6.376659 

5.623341 

3.342211 

7.657789 

3.267142 

6.732858 

55 

5.614811 

30.38519 

23.86294 

0.137055 

8.372963 

3.627037 

4.388539 

6.611461 

4.289968 

5.710032 

56 

4.933371 

31.06663 

20.96683 

3.033174 

7.356781 

4.643219 

3.855925 

7.144075 

3.769317 

6.230683 

57 

4.168595 

31.8314 

17.71653 

6.283471 

6.216326 

5.783674 

3.258176 

7.741824 

3.184994 

6.815006 

58 

3.60049 

32.39951 

15.30208 

8.697916 

5.369152 

6.630848 

2.814145 

8.185855 

2.750937 

7.249063 

59 

4.840475 

31.15953 

20.57202 

3.427982 

7.218252 

4.781748 

3.783317 

7.216683 

3.69834 

6.30166 

60 

5.485656 

30.51434 

23.31404 

0.685962 

8.180364 

3.819636 

4.287591 

6.712409 

4.191288 

5.808712 

61 

4.500473 

31.49953 

19.12701 

4.872988 

6.711232 

5.288768 

3.517572 

7.482428 

3.438564 

6.561436 

62 

5.394895 

30.6051 

22.9283 

1.071696 

8.045019 

3.954981 

4.216653 

6.783347 

4.121942 

5.878058 

MIN 

2.98 

30.31 

12.65 

0.00 

4.44 

3.52 

2.33 

6.56 

2.27 

5.66 

MAX 

5.69 

33.02 

24.17 

11.35 

8.48 

7.56 

4.44 

8.67 

4.34 

7.73 

MEAN 

4.27 

31.73 

18.16 

5.84 

6.37 

5.63 

3.34 

7.66 

3.27 

6.73 

MEDIAN 

4.25 

31.75 

18.05 

5.95 

6.33 

5.67 

3.32 

7.68 

3.24 

6.76 
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APPENDIX  E.  BASE  ‘C’  TRANSIT  AND  ON-STATION 

STATISTICS 


NON-PENETRATING  SCENARIO 

C 

MISSION 

RQ-4 

RQ-1 

RC-135 

U-2 

AREA 

TRANS. 

ON-STA 

TRANS. 

ON-STA 

TRANS. 

ON-STA 

TRANS. 

ON-STA 

1 

3.971865 

32.02814 

16.88043 

7.119575 

5.922956 

6.077044 

3.104411 

7.895589 

3.034683 

6.965317 

2 

3.527194 

32.47281 

14.99058 

9.009423 

5.259851 

6.740149 

2.756857 

8.243143 

2.694935 

7.305065 

3 

3.114286 

32.88571 

13.23572 

10.76428 

4.644111 

7.355889 

2.434128 

8.565872 

2.379455 

7.620545 

4 

2.735166 

33.26483 

1 1 .62445 

12.37555 

4.078756 

7.921244 

2.137807 

8.862193 

2.089789 

7.910211 

5 

2.38478 

33.61522 

10.13532 

13.86468 

3.556251 

8.443749 

1 .863945 

9.136055 

1.822079 

8.177921 

6 

2.07772 

33.92228 

8.830309 

15.16969 

3.098354 

8.901646 

1 .623947 

9.376053 

1.587471 

8.412529 

7 

1.835853 

34.16415 

7.802376 

16.19762 

2.737676 

9.262324 

1 .434904 

9.565096 

1 .402674 

8.597326 

8 

2.07772 

33.92228 

8.830309 

15.16969 

3.098354 

8.901646 

1 .623947 

9.376053 

1.587471 

8.412529 

9 

2.38478 

33.61522 

10.13532 

13.86468 

3.556251 

8.443749 

1 .863945 

9.136055 

1.822079 

8.177921 

10 

2.735166 

33.26483 

1 1 .62445 

12.37555 

4.078756 

7.921244 

2.137807 

8.862193 

2.089789 

7.910211 

11 

3.114286 

32.88571 

13.23572 

10.76428 

4.644111 

7.355889 

2.434128 

8.565872 

2.379455 

7.620545 

12 

3.470784 

32.52922 

14.75083 

9.24917 

5.17573 

6.82427 

2.712766 

8.287234 

2.651835 

7.348165 

13 

3.867275 

32.13272 

16.43592 

7.564079 

5.76699 

6.23301 

3.022664 

7.977336 

2.954772 

7.045228 

23 

4.416535 

31.58346 

18.77027 

5.229726 

6.586061 

5.413939 

3.451966 

7.548034 

3.374431 

6.625569 

MIN 

1.84 

31.58 

7.80 

2.74 

5.41 

1.43 

7.55 

1.40 

6.63 

MAX 

4.42 

34.16 

18.77 

6.59 

9.26 

3.45 

9.57 

3.37 

8.60 

MEAN 

2.98 

33.02 

12.66 

11.34 

4.44 

7.56 

2.33 

8.67 

2.28 

7.72 

MEDIAN 

2.92 

33.08 

12.43 

11.57 

4.36 

7.64 

2.29 

8.71 

2.23 

7.77 

PENETRATING  SCENARIO 

C 


MISSION 

RQ-4 

RQ-1 

RC-135 

U-2 

AREA 

TRANS. 

ON-STA 

TRANS. 

ON-STA 

TRANS. 

ON-STA 

TRANS. 

ON-STA 

1 

3.867275 

32.13272 

16.43592 

7.564079 

5.76699 

6.23301 

3.022664 

7.977336 

2.954772 

7.045228 

2 

3.470784 

32.52922 

14.75083 

9.24917 

5.17573 

6.82427 

2.712766 

8.287234 

2.651835 

7.348165 

3 

3.114286 

32.88571 

13.23572 

10.76428 

4.644111 

7.355889 

2.434128 

8.565872 

2.379455 

7.620545 

4 

2.735166 

33.26483 

1 1 .62445 

12.37555 

4.078756 

7.921244 

2.137807 

8.862193 

2.089789 

7.910211 

5 

2.38478 

33.61522 

10.13532 

13.86468 

3.556251 

8.443749 

1 .863945 

9.136055 

1.822079 

8.177921 

6 

2.07772 

33.92228 

8.830309 

15.16969 

3.098354 

8.901646 

1 .623947 

9.376053 

1.587471 

8.412529 

7 

1.835853 

34.16415 

7.802376 

16.19762 

2.737676 

9.262324 

1 .434904 

9.565096 

1.402674 

8.597326 

8 

2.07772 

33.92228 

8.830309 

15.16969 

3.098354 

8.901646 

1 .623947 

9.376053 

1.587471 

8.412529 

9 

2.38478 

33.61522 

10.13532 

13.86468 

3.556251 

8.443749 

1 .863945 

9.136055 

1.822079 

8.177921 

10 

2.735166 

33.26483 

1 1 .62445 

12.37555 

4.078756 

7.921244 

2.137807 

8.862193 

2.089789 

7.910211 

11 

3.114286 

32.88571 

13.23572 

10.76428 

4.644111 

7.355889 

2.434128 

8.565872 

2.379455 

7.620545 

12 

3.470784 

32.52922 

14.75083 

9.24917 

5.17573 

6.82427 

2.712766 

8.287234 

2.651835 

7.348165 

13 

3.867275 

32.13272 

16.43592 

7.564079 

5.76699 

6.23301 

3.022664 

7.977336 

2.954772 

7.045228 

24 

4.273261 

31.72674 

18.16136 

5.838639 

6.372407 

5.627593 

3.339983 

7.660017 

3.264964 

6.735036 

51 

3.717666 

32.28233 

15.80008 

8.199919 

5.543888 

6.456112 

2.90573 

8.09427 

2.840464 

7.159536 

52 

4.221248 

31.77875 

17.9403 

6.059697 

6.294843 

5.705157 

3.299329 

7.700671 

3.225223 

6.774777 

53 

2.899745 

33.10026 

12.32392 

11.67608 

4.324181 

7.675819 

2.266442 

8.733558 

2.215535 

7.784465 

54 

3.526173 

32.47383 

14.98623 

9.013767 

5.258327 

6.741673 

2.756058 

8.243942 

2.694154 

7.305846 

55 

4.764325 

31.23568 

20.24838 

3.75162 

7.104695 

4.895305 

3.723798 

7.276202 

3.640158 

6.359842 

56 

3.970271 

32.02973 

16.87365 

7.12635 

5.920579 

6.079421 

3.103165 

7.896835 

3.033465 

6.966535 

57 

3.017406 

32.98259 

12.82397 

11.17603 

4.49964 

7.50036 

2.358406 

8.641594 

2.305434 

7.694566 

58 

2.223352 

33.77665 

9.449244 

14.55076 

3.315524 

8.684476 

1 .737773 

9.262227 

1.698741 

8.301259 

59 

3.526173 

32.47383 

14.98623 

9.013767 

5.258327 

6.741673 

2.756058 

8.243942 

2.694154 

7.305846 

60 

4.221248 

31.77875 

17.9403 

6.059697 

6.294843 

5.705157 

3.299329 

7.700671 

3.225223 

6.774777 

61 

2.899745 

33.10026 

12.32392 

11.67608 

4.324181 

7.675819 

2.266442 

8.733558 

2.215535 

7.784465 

62 

3.717666 

32.28233 

15.80008 

8.199919 

5.543888 

6.456112 

2.90573 

8.09427 

2.840464 

7.159536 

MIN 

1.84 

31.24 

7.80 

3.75 

2.74 

4.90 

1.43 

7.28 

1.40 

6.36 

MAX 

4.76 

34.16 

20.25 

16.20 

7.10 

9.26 

3.72 

9.57 

3.64 

8.60 

MEAN 

3.24 

32.76 

13.75 

10.25 

4.82 

7.18 

2.53 

8.47 

2.47 

7.53 

MEDIAN 

3.29 

32.71 

13.99 

10.01 

4.91 

7.09 

2.57 

8.43 

2.52 

7.48 
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APPENDIX  F.  BASE  ‘D’  TRANSIT  AND  ON-STATION 

STATISTICS 


NON-PENETRATING  SCENARIO 

MISSION 

RQ-4 

RQ-1 

BSK5H 

RC-135 

U-2 

AREA 

TRANS. 

ON-STA 

TRANS. 

ON-STA 

TRANS. 

ON-STA 

TRANS. 

ON-STA 

1 

5.145959 

30.85404 

21.87032 

2.129675 

7.673798 

4.326202 

4.022084 

6.977916 

3.931744 

6.068256 

2 

4.701288 

31.29871 

19.98048 

4.019524 

7.010693 

4.989307 

3.674529 

7.325471 

3.591996 

6.408004 

3 

4.28838 

31.71162 

18.22562 

5.774384 

6.394953 

5.605047 

3.3518 

7.6482 

3.276515 

6.723485 

4 

3.828111 

32.17189 

16.26947 

7.730528 

5.708587 

6.291413 

2.992053 

8.007947 

2.924849 

7.075151 

5 

3.367842 

32.63216 

14.31333 

9.686673 

5.02222 

6.97778 

2.632307 

8.367693 

2.573182 

7.426818 

6 

2.907572 

33.09243 

12.35718 

1 1 .64282 

4.335854 

7.664146 

2.27256 

8.72744 

2.221516 

7.778484 

7 

2.447303 

33.5527 

10.40104 

13.59896 

3.649487 

8.350513 

1.912813 

9.087187 

1.86985 

8.13015 

8 

1.987132 

34.01287 

8.445312 

15.55469 

2.963267 

9.036733 

1.553143 

9.446857 

1.518258 

8.481742 

9 

1.52702 

34.47298 

6.489837 

17.51016 

2.277136 

9.722864 

1.19352 

9.80648 

1.166712 

8.833288 

10 

1 .067044 

34.93296 

4.534939 

19.46506 

1.591207 

10.40879 

0.834002 

10.166 

0.81527 

9.18473 

11 

0.607513 

35.39249 

2.581931 

21.41807 

0.905941 

11.09406 

0.474833 

10.52517 

0.464167 

9.535833 

12 

0.596622 

35.40338 

2.535642 

21.46436 

0.889699 

11.1103 

0.46632 

10.53368 

0.455846 

9.544154 

13 

0.850396 

35.1496 

3.614183 

20.38582 

1.268134 

10.73187 

0.66467 

10.33533 

0.649741 

9.350259 

24 

5.590629 

30.40937 

23.76017 

0.239827 

8.336903 

3.663097 

4.369638 

6.630362 

4.271492 

5.728508 

MIN 

0.60 

30.41 

0.24 

0.89 

3.66 

0.47 

6.63 

MAX 

5.59 

35.40 

21.46 

8.34 

11.11 

4.37 

10.53 

MEAN 

2.78 

33.22 

11.81 

12.19 

4.14 

7.86 

2.17 

8.83 

MEDIAN 

2.68 

33.32 

11.38 

12.62 

3.99 

8.01 

2.09 

8.91 

PENETRATING  SCENARIO 

D 


MISSION 

RQ-4 

RQ-1 

RC-135 

U-2 

AREA 

TRANS. 

ON-STA 

TRANS. 

ON-STA 

TRANS. 

ON-STA 

TRANS. 

ON-STA 

1 

3.932977 

32.06702 

16.71515 

7.284848 

5.864966 

6.135034 

3.074017 

7.925983 

3.004971 

6.995029 

2 

3.886012 

32.11399 

16.51555 

7.484448 

5.794931 

6.205069 

3.037309 

7.962691 

2.969088 

7.030912 

3 

3.887699 

32.1123 

16.52272 

7.477278 

5.797446 

6.202554 

3.038628 

7.961372 

2.970377 

7.029623 

4 

3.492483 

32.50752 

14.84305 

9.156949 

5.208088 

6.791912 

2.729726 

8.270274 

2.668414 

7.331586 

5 

3.115182 

32.88482 

13.23952 

10.76048 

4.645447 

7.354553 

2.434828 

8.565172 

2.380139 

7.619861 

6 

2.763146 

33.23685 

11.74337 

12.25663 

4.120481 

7.879519 

2.159676 

8.840324 

2.111168 

7.888832 

7 

2.447303 

33.5527 

10.40104 

13.59896 

3.649487 

8.350513 

1.912813 

9.087187 

1.86985 

8.13015 

8 

1.987132 

34.01287 

8.445312 

15.55469 

2.963267 

9.036733 

1.553143 

9.446857 

1.518258 

8.481742 

9 

1.52702 

34.47298 

6.489837 

17.51016 

2.277136 

9.722864 

1.19352 

9.80648 

1.166712 

8.833288 

10 

1 .067044 

34.93296 

4.534939 

19.46506 

1.591207 

10.40879 

0.834002 

10.166 

0.81527 

9.18473 

11 

0.607513 

35.39249 

2.581931 

21.41807 

0.905941 

11.09406 

0.474833 

10.52517 

0.464167 

9.535833 

12 

0.596622 

35.40338 

2.535642 

21.46436 

0.889699 

11.1103 

0.46632 

10.53368 

0.455846 

9.544154 

13 

0.850396 

35.1496 

3.614183 

20.38582 

1.268134 

10.73187 

0.66467 

10.33533 

0.649741 

9.350259 

24 

4.02877 

31.97123 

17.12227 

6.877728 

6.007815 

5.992185 

3.148888 

7.851112 

3.078161 

6.921839 

51 

3.551118 

32.44888 

15.09225 

8.907749 

5.295527 

6.704473 

2.775555 

8.224445 

2.713214 

7.286786 

52 

3.178579 

32.82142 

13.50896 

10.49104 

4.739986 

7.260014 

2.484379 

8.515621 

2.428577 

7.571423 

53 

2.89696 

33.10304 

12.31208 

1 1 .68792 

4.320028 

7.679972 

2.264265 

8.735735 

2.213408 

7.786592 

54 

2.773512 

33.22649 

1 1 .78743 

12.21257 

4.135939 

7.864061 

2.167778 

8.832222 

2.119088 

7.880912 

55 

2.928327 

33.07167 

12.44539 

11.55461 

4.366803 

7.633197 

2.288782 

8.711218 

2.237373 

7.762627 

56 

2.485783 

33.51422 

10.56458 

13.43542 

3.706869 

8.293131 

1 .942889 

9.057111 

1.89925 

8.10075 

57 

2.229016 

33.77098 

9.473319 

14.52668 

3.323971 

8.676029 

1 .7422 

9.2578 

1 .703069 

8.296931 

58 

2.312321 

33.68768 

9.827362 

14.17264 

3.448197 

8.551803 

1.807311 

9.192689 

1.766717 

8.233283 

59 

1.858833 

34.14117 

7.90004 

16.09996 

2.771944 

9.228056 

1 .452865 

9.547135 

1.420232 

8.579768 

60 

2.028526 

33.97147 

8.621235 

15.37876 

3.024995 

8.975005 

1 .585497 

9.414503 

1.549885 

8.450115 

61 

1 .550462 

34.44954 

6.589464 

17.41054 

2.312093 

9.687907 

1.211842 

9.788158 

1.184623 

8.815377 

62 

1.145201 

34.8548 

4.867105 

19.1329 

1 .707756 

10.29224 

0.89509 

10.10491 

0.874985 

9.125015 

MIN 

0.60 

2.54 

6.88 

0.89 

5.99 

0.47 

7.85 

0.46 

6.92 

MAX 

4.03 

17.12 

21.46 

6.01 

11.11 

3.15 

10.53 

3.08 

9.54 

MEAN 

2.43 

10.32 

13.68 

3.62 

8.38 

1.90 

9.10 

1.86 

8.14 

MEDIAN 

2.47 

33.53 

10.48 

13.52 

3.68 

8.32 

1.93 

9.07 

1.88 

8.12 
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APPENDIX  G.  NON-PENETRATING  COVERAGE  MODEL 


Summary  of  Fit 

RSquare  0.780377 

RSquare  Adj  0.763686 

Root  Mean  Square  Error  0.008541 

Mean  of  Response  0.485819 

Observations  (or  Sum  Wgts)  270 


Analysis  of  Variance 


Source  DF  Sum  of  Squares 

Model  19  0.06479516 

Error  250  0.01823541 

C.  Total  269  0.08303057 

Mean  Square 
0.003410 

0.000073 

F  Ratio 

46.7534 

Prob  >  F 

<.0001  * 

Parameter  Estimates 

Term 

Estimate 

Std  Error 

t  Ratio 

Prob>|t| 

Intercept 

0.4762323 

0.002843 

167.51 

<.0001  * 

ARQ-4 

-0.002767 

0.000534 

-5.18 

<.0001  * 

A  RQ-1 

-0.000885 

0.000285 

-3.10 

0.0021* 

A  RC-135 

0.0039884 

0.000535 

7.45 

<.0001  * 

AU-2 

0.0056309 

0.000536 

10.50 

<.0001  * 

B  RQ-4 

-0.002767 

0.000541 

-5.12 

<.0001  * 

B  RC-135 

-0.000835 

0.000538 

-1.55 

0.1219 

B  U-2 

0.0035851 

0.000534 

6.71 

<.0001  * 

C  RQ-4 

-0.002511 

0.000531 

-4.73 

<.0001  * 

C  U-2 

0.0058404 

0.000531 

10.99 

<.0001  * 

D  RQ-4 

-0.001957 

0.000529 

-3.70 

0.0003* 

D  U-2 

0.0074843 

0.000532 

14.07 

<.0001  * 

OPT  INT 

-0.004388 

0.000361 

-12.15 

<.0001  * 

(A  RQ-4-1 .52963)*(B  RC-1 35-1 .52963) 

-0.002434 

0.000522 

-4.66 

<.0001  * 

(A  RC-1 35-1 ,52963)*(D  RQ-4-1 .52963) 

-0.001769 

0.000518 

-3.41 

0.0007* 

(B  RC-1 35-1 ,52963)*(D  RQ-4-1 .52963) 

-0.001793 

0.000535 

-3.35 

0.0009* 

(C  RQ-4-1 .52963)*(D  U-2-1 .52963) 

-0.002549 

0.000504 

-5.05 

<.0001  * 

(B  RQ-4-1 .52963)*(B  RQ-4-1 .52963) 

0.001986 

0.000574 

3.46 

0.0006* 

(C  U-2-1 .52963)*(C  U-2-1 .52963) 

-0.001624 

0.00057 

-2.85 

0.0047* 

(D  RQ-4-1 .52963)*(D  RQ-4-1 .52963) 

0.0030399 

0.000559 

5.44 

<.0001  * 

Scaled  Estimates 


Continuous  factors  centered  by  mean,  scaled  by  range/2 
Term  Scaled  Estimate 


Intercept 

0.4823027 

ARQ-4 

-0.00415 

A  RQ-1 

-0.002655 

ARC-135 

0.0059826 

AU-2 

0.0084464 

B  RQ-4 

-0.00415 

B  RC-135 

-0.001253 

B  U-2 

0.0053776 

C  RQ-4 

-0.003766 

C  U-2 

0.0087605 

D  RQ-4 

-0.002936 

D  U-2 

0.0112265 

OPT  INT 

-0.011061 

(A  RQ-4-1 ,52963)*(B  RC-1 35-1 .52963) 

-0.005477 

(A  RC-1 35-1 ,52963)*(D  RQ-4-1 .52963) 

-0.00398 

(B  RC-1 35-1 ,52963)*(D  RQ-4-1 .52963) 

-0.004035 

(C  RQ-4-1 ,52963)*(D  U-2-1 .52963) 

-0.005736 

(B  RQ-4-1 ,52963)*(B  RQ-4-1 .52963) 

0.0044685 

(C  U-2-1 ,52963)*(C  U-2-1 .52963) 

-0.003654 

(D  RQ-4-1 ,52963)*(D  RQ-4-1 .52963) 

0.0068398 

Std  Error 
0.001085 
0.000802 
0.000856 
0.000803 
0.000805 
0.000811 
0.000807 
0.000801 
0.000796 
0.000797 
0.000793 
0.000798 
0.00091 
0.001175 
0.001166 
0.001204 
0.001135 
0.001292 
0.001282 
0.001258 


t  Ratio 

Prob>|t| 

444.69 

0.0000* 

-5.18 

<.0001  * 

-3.10 

0.0021  * 

7.45 

<.0001  * 

10.50 

<.0001  * 

-5.12 

<.0001  * 

-1.55 

0.1219 

6.71 

<.0001  * 

-4.73 

<.0001  * 

10.99 

<.0001  * 

-3.70 

0.0003* 

14.07 

<.0001  * 

-12.15 

<.0001  * 

-4.66 

<.0001  * 

-3.41 

0.0007* 

-3.35 

0.0009* 

-5.05 

<.0001  * 

3.46 

0.0006* 

-2.85 

0.0047* 

5.44 

<.0001  * 
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APPENDIX  H.  PENETRATING  COVERAGE  MODEL 


Summary  of  Fit 

RSquare  0.709558 

RSquare  Adj  0.677 1 53 

Root  Mean  Square  Error  0.031621 

Mean  of  Response  0.592963 

Observations  (or  Sum  Wgts)  270 


Analysis  of  Variance 


Source  DF  Sum  of  Squares  Mean  Square 

Model  27  0.59115961  0.021895 

Error  242  0.24197826  0.001000 

C.  Total  269  0.83313787 

F  Ratio 

21.8968 

Prob  >  F 

<.0001* 

Parameter  Estimates 

Term 

Estimate 

Std  Error 

t  Ratio 

Prob>|t| 

Intercept 

0.3606258 

0.011655 

30.94 

<.0001* 

ARQ-4 

0.0054156 

0.001988 

2.72 

0.0069* 

ARQ-1 

0.0065338 

0.001058 

6.17 

<.0001* 

A  P-3 

0.0061525 

0.001063 

5.79 

<.0001* 

AU-2 

0.0059377 

0.001998 

2.97 

0.0033* 

B  RQ-4 

0.0080442 

0.00197 

4.08 

<.0001* 

B  P-3 

0.003962 

0.001046 

3.79 

0.0002* 

B  U-2 

0.0064787 

0.001974 

3.28 

0.0012* 

C  RQ-4 

0.0107559 

0.001982 

5.43 

<.0001* 

CP-3 

0.0022811 

0.001052 

2.17 

0.0311* 

C  U-2 

0.0050889 

0.001977 

2.57 

0.0107* 

D  RQ-4 

0.0063526 

0.001993 

3.19 

0.0016* 

D  RQ-1 

0.003488 

0.001069 

3.26 

0.0013* 

DP-3 

0.0081541 

0.001063 

7.67 

<.0001* 

D  RC-135 

0.0071914 

0.002009 

3.58 

0.0004* 

DU-2 

0.0136722 

0.002008 

6.81 

<.0001* 

OPT  INT 

0.0139716 

0.001338 

10.45 

<.0001* 

(A  RQ-4-1 ,52963)*(OPT  INT-3.46986) 

-0.003346 

0.001438 

-2.33 

0.0208* 

(A  RQ-1  -3.05926)*(D  U-2-1 .52963) 

0.0032962 

0.001103 

2.99 

0.0031* 

(A  P-3-3. 05926)*(C  RQ-4-1 .52963) 

0.0009105 

0.000949 

0.96 

0.3383 

(A  U-2-1 ,52963)*(D  RQ-4-1 .52963) 

-0.004324 

0.001949 

-2.22 

0.0274* 

(B  RQ-4-1 ,52963)*(B  U-2-1 .52963) 

-0.003686 

0.002221 

-1.66 

0.0983 

(C  RQ-4-1 ,52963)*(OPT  INT-3.46986) 

-0.003959 

0.001359 

-2.91 

0.0039* 

(C  P-3-3. 05926)*(C  U-2-1 .52963) 

-0.001152 

0.001217 

-0.95 

0.3448 

(D  RQ-1  -3.05926)*(D  RC-1 35-1 .52963) 

-0.003878 

0.001108 

-3.50 

0.0005* 

(D  P-3-3. 05926)*(OPT  INT-3.46986) 

0.0006726 

0.000747 

0.90 

0.3688 

(B  P-3-3. 05926)*(B  P-3-3.05926) 

-0.001511 

0.000673 

-2.25 

0.0256* 

(D  U-2-1 ,52963)*(D  U-2-1 .52963) 

-0.008468 

0.00225 

-3.76 

0.0002* 

Scaled  Estimates 


Continuous  factors  centered  by  mean,  scaled  by  range/2 


Term 

Scaled  Estimate 

Intercept 

0.6080798 

ARQ-4 

0.0081234 

ARQ-1 

0.0196014 

A  P-3 

0.0184576 

AU-2 

0.0089066 

B  RQ-4 

0.0120663 

B  P-3 

0.011886 

B  U-2 

0.009718 

C  RQ-4 

0.0161338 

C  P-3 

0.0068433 

C  U-2 

0.0076333 

D  RQ-4 

0.0095289 

D  RQ-1 

0.010464 

D  P-3 

0.0244623 

D  RC-135 

0.010787 

D  U-2 

0.0205083 

OPT  INT 

0.0352201 

(A  RQ-4-1 ,52963)*(OPT  INT-3.46986) 

-0.01265 

(A  RQ-1  -3.05926)*(D  U-2-1 .52963) 

0.0148331 

(A  P-3-3. 05926)*(C  RQ-4-1 .52963) 

0.0040974 

(A  U-2-1 ,52963)*(D  RQ-4-1 .52963) 

-0.009729 

(B  RQ-4-1 ,52963)*(B  U-2-1 .52963) 

-0.008293 

(C  RQ-4-1 ,52963)*(OPT  INT-3.46986) 

-0.014971 

(C  P-3-3. 05926)*(C  U-2-1 .52963) 

-0.005185 

(D  RQ-1  -3.05926)*(D  RC-1 35-1 .52963) 

-0.017453 

(D  P-3-3. 05926)*(OPT  INT-3.46986) 

0.0050865 

(B  P-3-3. 05926)*(B  P-3-3.05926) 

-0.013603 

(D  U-2-1 .52963)*(D  U-2-1 .52963) 

-0.019052 

Std  Error 
0.003295 
0.002981 
0.003174 
0.003189 
0.002997 
0.002954 
0.003139 
0.002962 
0.002973 
0.003156 
0.002966 
0.002989 
0.003207 
0.00319 
0.003013 
0.003012 
0.003372 
0.005438 
0.004963 
0.004271 
0.004384 
0.004998 
0.005138 
0.005477 
0.004984 
0.00565 
0.006058 
0.005062 


t  Ratio 
184.53 
2.72 

6.17 

5.79 
2.97 
4.08 

3.79 
3.28 
5.43 

2.17 

2.57 
3.19 
3.26 
7.67 

3.58 
6.81 

10.45 

-2.33 

2.99 

0.96 

-2.22 

-1.66 

-2.91 

-0.95 

-3.50 

0.90 

-2.25 

-3.76 


Prob>|t| 

<.0001* 

0.0069* 

<.0001* 

<.0001* 

0.0033* 

<.0001* 

0.0002* 

0.0012* 

<.0001* 

0.0311* 

0.0107* 

0.0016* 

0.0013* 

<.0001* 

0.0004* 

<.0001* 

<.0001* 

0.0208* 

0.0031* 

0.3383 

0.0274* 

0.0983 

0.0039* 

0.3448 

0.0005* 

0.3688 

0.0256* 

0.0002* 
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Partition  for  Coverage 
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